Feat/intent responder ru #99
Merged
+1,886
−47
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…/intent_responder_RU
…/intent_responder_RU
…/intent_responder_RU
…/intent_responder_RU
…/intent_responder_RU # Conflicts: # assistant_dists/dream_russian/docker-compose.override.yml # assistant_dists/dream_russian/test.yml # tests/runtests_russian.sh
mtalimanchuk
approved these changes
Mar 3, 2022
|
convert-reddit and sentseg пока не заменены на русские версии и я не уверена, что у нас будут для них замены. поэтому я их пока убрала вообще из ветки feat.russian_baseline из компонент русской версии. |
dilyararimovna
added a commit
that referenced
this pull request
Jun 28, 2022
* feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы
dilyararimovna
added a commit
that referenced
this pull request
Jun 30, 2022
* feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com>
dariamitciuk
added a commit
to dariamitciuk/dream_mini
that referenced
this pull request
Aug 31, 2022
* fix/slot extr conf (deeppavlov#156) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (deeppavlov#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (deeppavlov#160) * Feat/transformers intent catcher (deeppavlov#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (deeppavlov#164) * Dream mini (deeppavlov#161) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (deeppavlov#163) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (deeppavlov#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (deeppavlov#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (deeppavlov#172) * Fix/protobuf version (deeppavlov#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (deeppavlov#176) * feat: docker compose with main components * Feat/tests russian (deeppavlov#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (deeppavlov#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (deeppavlov#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (deeppavlov#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (deeppavlov#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (deeppavlov#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (deeppavlov#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (deeppavlov#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (deeppavlov#97) * Fix requirements.txt (deeppavlov#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (deeppavlov#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (deeppavlov#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (deeppavlov#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (deeppavlov#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (deeppavlov#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (deeppavlov#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (deeppavlov#127) * Feat/spacy lemmatizer (deeppavlov#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (deeppavlov#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (deeppavlov#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (deeppavlov#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (deeppavlov#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (deeppavlov#183) * Feat/multilingual ner (deeppavlov#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (deeppavlov#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (deeppavlov#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (deeppavlov#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (deeppavlov#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (deeppavlov#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com>
dariamitciuk
added a commit
to dariamitciuk/dream_mini
that referenced
this pull request
Sep 16, 2022
* fix/slot extr conf (deeppavlov#156) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (deeppavlov#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (deeppavlov#160) * Feat/transformers intent catcher (deeppavlov#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (deeppavlov#164) * Dream mini (deeppavlov#161) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (deeppavlov#163) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (deeppavlov#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (deeppavlov#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (deeppavlov#172) * Fix/protobuf version (deeppavlov#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (deeppavlov#176) * feat: docker compose with main components * Feat/tests russian (deeppavlov#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (deeppavlov#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (deeppavlov#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (deeppavlov#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (deeppavlov#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (deeppavlov#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (deeppavlov#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (deeppavlov#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (deeppavlov#97) * Fix requirements.txt (deeppavlov#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (deeppavlov#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (deeppavlov#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (deeppavlov#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (deeppavlov#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (deeppavlov#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (deeppavlov#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (deeppavlov#127) * Feat/spacy lemmatizer (deeppavlov#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (deeppavlov#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (deeppavlov#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (deeppavlov#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (deeppavlov#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (deeppavlov#183) * Feat/multilingual ner (deeppavlov#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (deeppavlov#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (deeppavlov#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (deeppavlov#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (deeppavlov#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (deeppavlov#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: prompts from dummy skill (deeppavlov#193) * Feat/sentence ranker as a service (deeppavlov#191) * feat: sentence ranker almost * feat: sentence ranker * fix: tests * fix: get scores * fix: codestyle * fix: reqs * fix: flask jsonify * fix: flask jsonify * fix: json types * fix: logs * fix: usage of single scores calcution * fix: codestyle * fix: codestyle * fix: refactor * fix: scores and curr_single_scores * fix: codestyle * feat: log * fix: model and test * fix: upd reqs for kg service (deeppavlov#195) * image captioning * update image captioning server.py * update image captioning server.py * add dream_multimodal * updates * updated pipeline Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com>
dilyararimovna
added a commit
to dariamitciuk/dream_mini
that referenced
this pull request
Sep 27, 2022
* fix/slot extr conf (deeppavlov#156) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (deeppavlov#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (deeppavlov#160) * Feat/transformers intent catcher (deeppavlov#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (deeppavlov#164) * Dream mini (deeppavlov#161) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (deeppavlov#163) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (deeppavlov#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (deeppavlov#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (deeppavlov#172) * Fix/protobuf version (deeppavlov#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (deeppavlov#176) * feat: docker compose with main components * Feat/tests russian (deeppavlov#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (deeppavlov#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (deeppavlov#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (deeppavlov#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (deeppavlov#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (deeppavlov#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (deeppavlov#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (deeppavlov#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (deeppavlov#97) * Fix requirements.txt (deeppavlov#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (deeppavlov#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (deeppavlov#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (deeppavlov#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (deeppavlov#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (deeppavlov#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (deeppavlov#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (deeppavlov#127) * Feat/spacy lemmatizer (deeppavlov#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (deeppavlov#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (deeppavlov#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (deeppavlov#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (deeppavlov#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (deeppavlov#183) * Feat/multilingual ner (deeppavlov#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (deeppavlov#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (deeppavlov#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (deeppavlov#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (deeppavlov#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (deeppavlov#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: prompts from dummy skill (deeppavlov#193) * Feat/sentence ranker as a service (deeppavlov#191) * feat: sentence ranker almost * feat: sentence ranker * fix: tests * fix: get scores * fix: codestyle * fix: reqs * fix: flask jsonify * fix: flask jsonify * fix: json types * fix: logs * fix: usage of single scores calcution * fix: codestyle * fix: codestyle * fix: refactor * fix: scores and curr_single_scores * fix: codestyle * feat: log * fix: model and test * fix: upd reqs for kg service (deeppavlov#195) * image captioning * update image captioning server.py * update image captioning server.py * add dream_multimodal * updates * updated pipeline Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com>
dilyararimovna
added a commit
that referenced
this pull request
Sep 28, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * update (#2) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (#183) * Feat/multilingual ner (#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com> * image captioning * image captioning * Image captioning (#4) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (#183) * Feat/multilingual ner (#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: prompts from dummy skill (#193) * Feat/sentence ranker as a service (#191) * feat: sentence ranker almost * feat: sentence ranker * fix: tests * fix: get scores * fix: codestyle * fix: reqs * fix: flask jsonify * fix: flask jsonify * fix: json types * fix: logs * fix: usage of single scores calcution * fix: codestyle * fix: codestyle * fix: refactor * fix: scores and curr_single_scores * fix: codestyle * feat: log * fix: model and test * fix: upd reqs for kg service (#195) * image captioning * update image captioning server.py * update image captioning server.py * add dream_multimodal * updates * updated pipeline Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com> * add: files * fix: docs line * fix: codestyule * fix: run command * fix: run command * fix itsdangerous requirements * image captioning * image captioning * Image captioning (#4) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna …
IgnatovFedor
pushed a commit
that referenced
this pull request
Dec 6, 2023
* feat: management distribution
* fix: prompt selector for roles
* first commit for meeting analysis
* working distribution, but no meeting analysis yet
* prototype files
* prompts
* dff_meeting_analysis_skill instead of prompted; llm-based everything
* working version of meeting analysis skill
* dff_meeting_analysis_skill with 4 nodes
* doc-processor annotator
* added saving previous meeting analysis results; links to them are written to bot attrs
* update roles
* fix for meeting analysis skill, now working
* document only for now, then will be deleted
* prompt for unabridged response selection
* refactor doc_processor, remove unnecessary funcs
* better prompts
* better skill description in components
* add llm-based-skill-selector to dist
* enable finding previously generated meeting analyses; better fallback
* 512 max_tokens for chatgpt in some cases
* enhance response selector prompt
* add dff_meeting_analysis_skill_formatter
* some fixes to cards and configs
* update readmes
* correct ports for doc processor; remove extra prompt
* codestyle
* codestyle
* fixes for Dilya
* enhanced checks
* typo
* codestyle + small fix for checks
* file moved to google drive
* remove extra print
* checking each file if processed; concatenating multiple files; two containers for doc-processor
* typo fix
* unique ids for files in data/, ids to paths in config
* delete transcript files
* codestyle
* fix: UIDs for files in data now working
* fixes in working with files
* codestyle
* fix error in getting related_files
* Revert "fix error in getting related_files"
This reverts commit 705e23897e9317e1ba24702b14e7c097da093dcd.
* working fix for bot_attrs_files
* remove document file
* numerous fixes for review
* codestyle
* bring some things to common
* even better funcs in common
* codestyle
* saving all processed docs in atts; saving candidate texts in adds of utt; link or path possible for processing from atts
* fixes for accidentally broken stuff
* some more fixes
* candidate texts to hyp attributes
* codestyle
* FILE_SERVER_TIMEOUT as arg
* GENERATIVE_SERVICE_URL as arg
* fix: formatters in pipeline_conf
* component card for vectorize_documents
* openai-chatgpt-long.json for document-qa-llm-skill
* openai-chatgpt-long.json for meeting-analysis-skill
* fix: timeouts and component card paths
* add regex for http check
* doc processor names in service_config files
* update getting envvars
* codestyle
* fix: remove envvars from everywhere
* fix: remove envvars from everywhere
* fixes: details in cards and pipeline
* fixes: details in cards and pipeline
* feat: special message if failed to process file from atts
* get token limit from service endpoint
* fix: better upload_document, try except inside func & enable both text and file upload in one func
* docstrings; also fix: detecting extension for links
* codestyle
* again codestyle
* update READMEs with dialog state info
* fix: add diff endpoints to doc-retriever readme
* fix: solve inconsistencies in cards and readmes
* fix: incorrect formatters in cards
* update ports to non-allocated ones
* fixes: everything acc to comments
* codestyle
* generalize file service url in another comment
* codestyle
* refactor attributes structure
* update readmes to include info about new attributes format
* fix: clean config; comment about format
* add comments; {FILE_SERVER_URL} instead of actual url
* comments and readmes
* implement storing doc for N_TURNS_FOR_DISCUSSION turns
* codestyle
* improve N_TURNS_FOR_DISCUSSION, implement only for doc-processor-from-atts
* better logging in doc-retriever
* codestyle
* more comments
* codestyle
* delete extra logs
* some more comments
* count n_steps_discussed in any case; put that to readme
* fix: n_steps_discussed in correct place
* fix: if file was processed earlier, take processed text from processed_documents
* if we get doc from somewhere, consider it good as new -> reset n_steps_discussed to 0
* codestyle
* update comments; fix logic of n_steps_discussed
* better comments
* fix: small fixes
* N_TURNS_FOR_DISCUSSION: -> N_TURNS_TO_KEEP_DOC
* N_TURNS_TO_KEEP_DOC in distribution files
* N_TURNS_TO_KEEP_DOC: 10 ->; also updates in readmes and comments
* codestyle
* comment about N_TURNS_TO_KEEP_DOC
* comment about N_TURNS_TO_KEEP_DOC
* fix: remove sentseg from management dist
* better descriptions for skills
* fix hyp format for dff_meeting_analysis_skill
* fixes: remove logs, improve skill description
* ensure unique ids everywhere; add dialog_id to file_id
* update skill selector: turn off doc-based skills when we don't have doc
* codestyle
* codestyle again
* remove one extra log
* now we can also process files from file server
* codestyle
* fix: is_container_running to response.py
* fix to prompts; also longer context for many services
* always turn on document-based wa skill
* codestyle
* add file exists check
* start adding question_answering default node
* node for question answering in meeting analysis skill; small change in llm-based-skill-selector
* codestyle
* condition file
* Dilya's fixes for skill-selector
* codestyle
* slightly improve prompt for response selector
* fix: chunks only split by newlines
* fix: no extra info in prompts; better response selector
* small fixes
* codestyle
* added list title
* codestyle
* codestyle
* moved is_container_running up
* fix: tags: selector
* add check if skill to add is in pipeline
* shorter prompt for response selector
* copy older dist with tf-idf qa as management_assistant_extended
* remove tf-idf qa skill from management assistant
* update description for meeting analysis skill
* remover doc-retriever from main distribution
* better guidance for qa
* feat: turn on dff_meeting_analysis_skill when it was used with the same doc before
* codestyle
* codestyle
* fix: only perform doc-related checks in skill selector if we actually have a doc in use
* fix: include situation when we don't have prev_skills or prev_docs in skill selector
* use gpt4 for meeting analysis skill
* feat: add progress by areas
* improve prompts
* gpt-4 response selector
* feat: weekly reports, draft
* improved prompts for showing titles
* huge timeout
* add re.DOTALL flag
* fix: regex for conditions
* now working with separate files in use
* update attributes format (for docs_in_use)
* update test files for new attribute format
* codestyle
* update annotator readmes
* update skill for new attributes format
* improve comments
* switch to chatgpt
* fixing conflicts from merge
* fix things lost during merge
* codestyle
* add some more accidentally lost info
* return accidentally lost change
* changes for Dilya
* filetype exception - remove logging
* remove sentry from utils.py
* flake8 improve work with exception; update info about meeting skill in extended dist
* update envvars
* remove unnecessary const
---------
Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-82-243.ec2.internal>
IgnatovFedor
added a commit
that referenced
this pull request
Dec 6, 2023
* feat: Azure OpenAI * fix: black * refactor: changed davinci3 to 2 * fix: tests * refactor: added missing newline * refactor: code formatting * fix: use .env_secret_azure for additional env vars for azure * fix: use .env_azure for public services * feat: azure api variables * fix: use .env_azure for public services * feat: created .env_secret_azure * fix: use .env_azure for management assistants * Feat/doc skills turnon logic to common (#94) * move doc skills logic to common; introduce it to desc based skill selector * turn on doc-based skills if we have doc in use for desc based skill selector; complex checks for llm based skill selector * remove dff_meeting_analysis_skill from automatically added skills * add comment about turning on doc based skills * add doc-skill turn on logic to universal llm-based skill selector; also fix the issue with activating all skills from pipeline if there is an exception * codestyle * remove extra list(set()) * fixes acc to Dilya * fix: skill selection logic with docs also * fix: codestyle * codestyle * remove N_TURNS_TO_KEEP_DOC from skill selector --------- Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/weekly with separate files (#99) * feat: management distribution * fix: prompt selector for roles * first commit for meeting analysis * working distribution, but no meeting analysis yet * prototype files * prompts * dff_meeting_analysis_skill instead of prompted; llm-based everything * working version of meeting analysis skill * dff_meeting_analysis_skill with 4 nodes * doc-processor annotator * added saving previous meeting analysis results; links to them are written to bot attrs * update roles * fix for meeting analysis skill, now working * document only for now, then will be deleted * prompt for unabridged response selection * refactor doc_processor, remove unnecessary funcs * better prompts * better skill description in components * add llm-based-skill-selector to dist * enable finding previously generated meeting analyses; better fallback * 512 max_tokens for chatgpt in some cases * enhance response selector prompt * add dff_meeting_analysis_skill_formatter * some fixes to cards and configs * update readmes * correct ports for doc processor; remove extra prompt * codestyle * codestyle * fixes for Dilya * enhanced checks * typo * codestyle + small fix for checks * file moved to google drive * remove extra print * checking each file if processed; concatenating multiple files; two containers for doc-processor * typo fix * unique ids for files in data/, ids to paths in config * delete transcript files * codestyle * fix: UIDs for files in data now working * fixes in working with files * codestyle * fix error in getting related_files * Revert "fix error in getting related_files" This reverts commit 705e23897e9317e1ba24702b14e7c097da093dcd. * working fix for bot_attrs_files * remove document file * numerous fixes for review * codestyle * bring some things to common * even better funcs in common * codestyle * saving all processed docs in atts; saving candidate texts in adds of utt; link or path possible for processing from atts * fixes for accidentally broken stuff * some more fixes * candidate texts to hyp attributes * codestyle * FILE_SERVER_TIMEOUT as arg * GENERATIVE_SERVICE_URL as arg * fix: formatters in pipeline_conf * component card for vectorize_documents * openai-chatgpt-long.json for document-qa-llm-skill * openai-chatgpt-long.json for meeting-analysis-skill * fix: timeouts and component card paths * add regex for http check * doc processor names in service_config files * update getting envvars * codestyle * fix: remove envvars from everywhere * fix: remove envvars from everywhere * fixes: details in cards and pipeline * fixes: details in cards and pipeline * feat: special message if failed to process file from atts * get token limit from service endpoint * fix: better upload_document, try except inside func & enable both text and file upload in one func * docstrings; also fix: detecting extension for links * codestyle * again codestyle * update READMEs with dialog state info * fix: add diff endpoints to doc-retriever readme * fix: solve inconsistencies in cards and readmes * fix: incorrect formatters in cards * update ports to non-allocated ones * fixes: everything acc to comments * codestyle * generalize file service url in another comment * codestyle * refactor attributes structure * update readmes to include info about new attributes format * fix: clean config; comment about format * add comments; {FILE_SERVER_URL} instead of actual url * comments and readmes * implement storing doc for N_TURNS_FOR_DISCUSSION turns * codestyle * improve N_TURNS_FOR_DISCUSSION, implement only for doc-processor-from-atts * better logging in doc-retriever * codestyle * more comments * codestyle * delete extra logs * some more comments * count n_steps_discussed in any case; put that to readme * fix: n_steps_discussed in correct place * fix: if file was processed earlier, take processed text from processed_documents * if we get doc from somewhere, consider it good as new -> reset n_steps_discussed to 0 * codestyle * update comments; fix logic of n_steps_discussed * better comments * fix: small fixes * N_TURNS_FOR_DISCUSSION: -> N_TURNS_TO_KEEP_DOC * N_TURNS_TO_KEEP_DOC in distribution files * N_TURNS_TO_KEEP_DOC: 10 ->; also updates in readmes and comments * codestyle * comment about N_TURNS_TO_KEEP_DOC * comment about N_TURNS_TO_KEEP_DOC * fix: remove sentseg from management dist * better descriptions for skills * fix hyp format for dff_meeting_analysis_skill * fixes: remove logs, improve skill description * ensure unique ids everywhere; add dialog_id to file_id * update skill selector: turn off doc-based skills when we don't have doc * codestyle * codestyle again * remove one extra log * now we can also process files from file server * codestyle * fix: is_container_running to response.py * fix to prompts; also longer context for many services * always turn on document-based wa skill * codestyle * add file exists check * start adding question_answering default node * node for question answering in meeting analysis skill; small change in llm-based-skill-selector * codestyle * condition file * Dilya's fixes for skill-selector * codestyle * slightly improve prompt for response selector * fix: chunks only split by newlines * fix: no extra info in prompts; better response selector * small fixes * codestyle * added list title * codestyle * codestyle * moved is_container_running up * fix: tags: selector * add check if skill to add is in pipeline * shorter prompt for response selector * copy older dist with tf-idf qa as management_assistant_extended * remove tf-idf qa skill from management assistant * update description for meeting analysis skill * remover doc-retriever from main distribution * better guidance for qa * feat: turn on dff_meeting_analysis_skill when it was used with the same doc before * codestyle * codestyle * fix: only perform doc-related checks in skill selector if we actually have a doc in use * fix: include situation when we don't have prev_skills or prev_docs in skill selector * use gpt4 for meeting analysis skill * feat: add progress by areas * improve prompts * gpt-4 response selector * feat: weekly reports, draft * improved prompts for showing titles * huge timeout * add re.DOTALL flag * fix: regex for conditions * now working with separate files in use * update attributes format (for docs_in_use) * update test files for new attribute format * codestyle * update annotator readmes * update skill for new attributes format * improve comments * switch to chatgpt * fixing conflicts from merge * fix things lost during merge * codestyle * add some more accidentally lost info * return accidentally lost change * changes for Dilya * filetype exception - remove logging * remove sentry from utils.py * flake8 improve work with exception; update info about meeting skill in extended dist * update envvars * remove unnecessary const --------- Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-82-243.ec2.internal> * fix: remove envvars to send from attributes (#102) * Feat/check before question answering node in meeting analysis (#104) * first commit for check before call LLM * condition for calling gpt4: WIP * condition for calling gpt4: WIP-2 * working check before qa node * docker container arg SHORT_GENERATIVE_SERVICE everuwhere; fix README * codestyle * update docs_in_use; add comment * move prompts to common * fix typo * Feat/summary length options (#105) * feat: length of summary now controllable * codestyle * flag re.IGNORECASE * gpt4 for response generation and selection in management assistant dists (#106) * replace chatgpt with gpt4 for response generation and selection in management assistant dists * add gpt4 container to management_assistant * also add to dev.yml * llm-based-response-selector-gpt4 * fixes acc to Dilya * feat: show up google api skill (#52) * feat: show up google api skill * fix: do not use envvars to send in google api skill * fix: timeout for google api skill * fix: do not wait for google api * fix: short_generative_service in correct Dockerfile (#107) * Feat/nice formatting (#110) * formatting: first commit * unify summary descriptions * formatting for titles completed * fix compose_variables; fix getting parts of report; fix summary length prompts; fix formatting * fix: verify=False for getting files * improve some prompts * working formatting * codestyle * add comments * formatting fixes * sent most of logic to utils * codestyle * fix: use .env_azure --------- Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> Co-authored-by: Nika Smilga <42929200+smilni@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-82-243.ec2.internal>
oserikov
pushed a commit
that referenced
this pull request
Nov 21, 2024
* feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com>
oserikov
pushed a commit
that referenced
this pull request
Nov 21, 2024
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * update (#2) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (#183) * Feat/multilingual ner (#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com> * image captioning * image captioning * Image captioning (#4) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (#183) * Feat/multilingual ner (#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: prompts from dummy skill (#193) * Feat/sentence ranker as a service (#191) * feat: sentence ranker almost * feat: sentence ranker * fix: tests * fix: get scores * fix: codestyle * fix: reqs * fix: flask jsonify * fix: flask jsonify * fix: json types * fix: logs * fix: usage of single scores calcution * fix: codestyle * fix: codestyle * fix: refactor * fix: scores and curr_single_scores * fix: codestyle * feat: log * fix: model and test * fix: upd reqs for kg service (#195) * image captioning * update image captioning server.py * update image captioning server.py * add dream_multimodal * updates * updated pipeline Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com> * add: files * fix: docs line * fix: codestyule * fix: run command * fix: run command * fix itsdangerous requirements * image captioning * image captioning * Image captioning (#4) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna …
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.