Briko Translation Service

Briko Translation Service Based on Tensorflow Transformer Translation Model

Introduction

This is a translation service built on Tensorflow transformer translation model (ver: r1.13.0).

It is part of our COVID-19 News Bot on Telegram Project, you can find more about this project here.

We have trained several translator models using the existing open-source corpus and this service can be used as an API server, based on Flask, to host these translator models. This server can be used in junction with applications such as a Telegram bot to provide multi-language machine translation capabilities.

While we do not dive into the details of the transformer architecture or the process of model training here, we highly recommend you to read this paper Attention is All You Need and the training walkthrough if you are interested and like to learn more about it.

System Environment

This repo is developed and tested on:

System	CUDA	Python	Tensorflow
Ubuntu 18.04.2 LTS	10.2	3.6.8	1.13.2

Walkthrough

Before you start, please clone this repository, preferably into your existing transformer rep:.

/path/to/models/offcial/transformer/

There are a few dependencies need to be installed, please run:

pip install -r requirements.txt

Please remember to modify config.py to match your setup environment. To start the server, run the following script:

./run_services.sh

• Please note that the model checkpoints as well as the vocab file are assumed to be placed in _PRODUCTION_MODEL_PATH.

Appendix

API Address

0.0.0.0:1234/translator

Translation Request:

{
    "msgType": "Translation",
    "msgID" : "BOT00012234",
    "sourceLang" : "en",
    "requestLang" : ["zh","fr"],
    "sourceContent" : "this is a test."
}

Response:

Successful:

{
    "msgType": "RespondTranslation",
    "msgID" : "TRANS00012",
    "msgIDRespondTo" : "BOT00012234",
    "msgFlag" : "success",
    "translationResults": {"fr": "C'est un test.", "zh": "\u8fd9 \u662f \u4e00 \u4e2a \u6d4b\u8bd5 \u3002"}
}

Failed:

{
    "msgType": "RespondTranslation",
    "msgID" : "TRANS00012",
    "msgIDRespondTo" : "BOT00012234",
    "msgFlag" : "failed",
    "translationResults" : ""
}

Status Check Request:

{
    "msgType": "StatusCheck",
    "msgID" : "BOT000002"
}

Healthy:

{
    "msgType": "StatusCheck",
    "msgID" : "TRANS00235",
    "msgIDRespondTo" : "BOT000002",
    "msgFlag" : "healthy"
}

Wrong Request Response:

{
    "msgType": "I have an apple.",
    "msgID" : "BOT00012234",
    "sourceLang" : "en",
    "requestLang" : "jp",
    "sourceContent" : "I have an apple."
}

Response:

{
    "msgType": "I have an apple.",
    "msgID" : "TRANS00235",
    "msgIDRespondTo" : "BOT00012234",
    "msgFlag" : "Wrong msg type"
}

License:

MIT