Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NEW] Integrate DEEPL translation service to RC core #12174

Merged
merged 25 commits into from
Aug 10, 2019

Conversation

vickyokrm
Copy link
Contributor

@vickyokrm vickyokrm commented Sep 26, 2018

Motivation

Google Translate is good, but there are other translation providers which are better - at least for particular languages / combinations.

What this adds

This PR changes will allow DEEPL translation provider to be used in the core Rocket.Chat.
More info about DeepL.
This also refactors the standard auto-translate package so that common code across translation provides (such as emoji- or url-handling) can be re-used easily.

@graywolf336
Copy link
Contributor

And now to turn these abstraction into an App item ;)

@vickyokrm vickyokrm reopened this Apr 30, 2019
@vickyokrm vickyokrm force-pushed the DEEPL-translation branch from 53ead3f to d0198cc Compare May 2, 2019 08:42
@vickyokrm vickyokrm force-pushed the DEEPL-translation branch from c6a0591 to a2632d2 Compare May 2, 2019 10:11
@geekgonecrazy
Copy link
Contributor

We should try to get this one in 1.1. Giving people options to send their text to some place other than google for translation... I think will be a huge win.

@graywolf336
Copy link
Contributor

@geekgonecrazy I agree :)

@vickyokrm vickyokrm force-pushed the DEEPL-translation branch from efa2d1a to 9a68dad Compare May 4, 2019 10:03
@vickyokrm
Copy link
Contributor Author

@geekgonecrazy How can I assist to get this feature included on the next release?

@mrsimpson
Copy link
Collaborator

External services are tricky to test-automate, so I manually verified that Google Translation still works as well:

  • Language names are being retrieved in the language of the user (getSupportedLanguages())
  • Messages are being translated to the chosen target language
  • Descriptions of message attachments are being translated

So imho, this is good to be merged now. Anyone volunteering?

@geekgonecrazy
Copy link
Contributor

@diegolmello I believe auto translate was just implemented in RN. How will this effect? Any feedback here on api especially

@mrsimpson mrsimpson requested a review from sampaiodiego August 9, 2019 06:22
@mrsimpson
Copy link
Collaborator

@geekgonecrazy This is a backend-feature, so no changes are expected on any consumer. Also the API for retrieving the translations has not been changed, so I can't see how this PR should effect the RN app.

However, I tested it (1.17.0, 10043) .
I could not see messages in the ReactNative App being shown as translated. Still, messages I wrote there were shown as translated on the browser. Therefore, I consider the changes @diegolmello was working on just did not make it into the Testflight-version I'm using.

@geekgonecrazy is there anything else preventing this feature from being merged? We do have some other translation related stuff on our backlog which depends on this, so it was really important to know it's in core

geekgonecrazy
geekgonecrazy previously approved these changes Aug 9, 2019
@geekgonecrazy geekgonecrazy added this to the 2.0.0 milestone Aug 9, 2019
@geekgonecrazy
Copy link
Contributor

Retriggering the build to make sure it gets past tests 🤞

Copy link
Collaborator

@mrsimpson mrsimpson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vickyokrm please check the setting used by the google translation provider!

@mrsimpson mrsimpson self-requested a review August 10, 2019 09:31
Copy link
Collaborator

@mrsimpson mrsimpson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@geekgonecrazy corrected it already ;)

@mrsimpson mrsimpson dismissed their stale review August 10, 2019 09:32

Aaron corrected it already

@geekgonecrazy geekgonecrazy merged commit 2c3c2dc into RocketChat:develop Aug 10, 2019
@geekgonecrazy geekgonecrazy deleted the DEEPL-translation branch August 10, 2019 09:43
Kailash0311 pushed a commit to WideChat/Rocket.Chat that referenced this pull request Aug 21, 2019
* [NEW] Service Account Admin Settings and Configuration files added

* Service Account Creation dialog added

* [NEW] Service Account Creation method

* Service Account owner username update method added

* Fixed CLI errors

* Fixed CLI errors

* Service Account creation heading fixed

* Service Account broadcast room callback added

* Service Account creation method refactored

* Service Account Callback completed

* Typos fixed

* CLI errors fixed

* [NEW] Service Account one-tap login complete

* Callbacks modified

* Service Accounts directory tab added

* Refactored creation method and added tests

* CLI errors fixed

* CLI errors fixed

* Bugs fixed

* [NEW] Service Accounts Login method

* Typo fixed

* CLI errors fixed

* CLI errors fixed

* [New] Service Account directory feature

* CLI errors fixed

* UsernameExists meteor method fixed

* Sync commit

* [NEW] Service Account subscription method added

* [NEW] Service Account Broadcast Feature Added

* [NEW] Service Account Broadcast Feature Added

* [NEW] Service account subscription sidenav type

* Broadcast Room name change handled

* Lint errors fixed

* getLoginToken method refactored

* Console statements removed

* Sidebar header permission modified

* Merge branch service-accounts

* Added service account directory search translation key

* Subscribers count added

* [NEW] Service Account sidenav type

* [FIX] Not showing local app on App Details (RocketChat#14894)

* Bump version to 1.2.1

* Unread counter added in popver

* Get linked service account method added

* Partial rate limiter added

* Added unread counter

* Fixed CLI errors

* Broadcast feature added

* [FIX] Custom status displayed on room leader panel (RocketChat#14958)

* [FIX] LDAP login with customField sync (RocketChat#14808)

Closes RocketChat#14661

* [FIX] Prevent error on trying insert message with duplicated id (RocketChat#14945)

* [FIX] OTR key icon missing on messages (RocketChat#14953)

* [FIX] Method `getUsersOfRoom` not returning offline users if limit is not defined (RocketChat#14753)

* [IMPROVE] Remove too specific helpers isFirefox() and isChrome() (RocketChat#14963)

* [FIX] Jump to message missing in Starred Messages (RocketChat#14949)

* [IMPROVE] Update tabs markup (RocketChat#14964)

* [FIX] Loading indicator positioning (RocketChat#14968)

* Remove broken markup

* Recover lost class

* [FIX] load more messages (RocketChat#14967)

* [FIX] eternal loading file list (RocketChat#14952)

* [FIX] 50 custom emoji limit (RocketChat#14951)

* Bump jquery from 3.3.1 to 3.4.0 in /packages/rocketchat-livech… (RocketChat#14922)

Bumps [jquery](https://github.com/jquery/jquery) from 3.3.1 to 3.4.0.
- [Release notes](https://github.com/jquery/jquery/releases)
- [Commits](jquery/jquery@3.3.1...3.4.0)

Signed-off-by: dependabot[bot] <support@github.com>

* [FIX] Allow storing the navigation history of unregistered Livechat visitors (RocketChat#14970)

* Remove validations before storing visitor navigation history.

* Removed unused imports.

* Update GPG key

* [FIX] Wrong label order on room settings (RocketChat#14960)

* [FIX] Chrome doesn't load additional search results when botto… (RocketChat#14965)

* fix scroll

* fix review

* ops

* [IMPROVEMENT] patch to improve emoji render (RocketChat#14722)

* path to improve emoji render

* Apply suggestions from code review

Co-Authored-By: Tasso Evangelista <tasso.evangelista@rocket.chat>

* Regression: patch to improve emoji render (RocketChat#14980)

This reverts commit b395b50.

* [FIX] Users staying online after logout (RocketChat#14966)

* Remove unused dependency (lokijs) (RocketChat#14973)

* Bump juice version to 5.2.0 (RocketChat#14974)

* Bump node-rsa version to 1.0.5 (RocketChat#14976)

* Bump photoswipe version to 4.1.3 (RocketChat#14977)

* Remove unused Meteor dependency (yasinuslu:blaze-meta) (RocketChat#14971)

* Bump marked from 0.5.2 to 0.6.1 (RocketChat#14969)

* Bump marked from 0.5.2 to 0.6.1

Bumps [marked](https://github.com/markedjs/marked) from 0.5.2 to 0.6.1.
- [Release notes](https://github.com/markedjs/marked/releases)
- [Commits](markedjs/marked@v0.5.2...v0.6.1)

Signed-off-by: dependabot[bot] <support@github.com>

* Update package-lock

* [IMPROVE] Federation routes RocketChat#14972

* [IMPROVE] Extract federation config to its own file (RocketChat#14992)

* Revert "[IMPROVE] Federation routes RocketChat#14972"

* Extract federation config logic to its own file

* Service account configurable approval process

* Improve Docker compose readability (RocketChat#14457)

* [FIX] Not being able to mention users with "all" and "here" usernames - do not allow users register that usernames (RocketChat#14468)

* [FIX] Custom User Status throttled by rate limiter (RocketChat#15001)

* [FIX] CustomOauth Identity Step errors displayed in HTML format (RocketChat#15000)

* Service account message alert added

* Lint errors fixed

* fixed browseChannels method

* fixed browseChannels method

* Service Account Notification fixed

* [IMPROVE] Add flag to identify remote federation users (RocketChat#15004)

* added isRemote flag to users

* set isRemote on peers collection
* moved some finds to its models
* use new raw model

* Add more indeces and model methods

* [FIX] Always displaying jumbomojis when using "marked" markdown (RocketChat#14861)

* relocate some of wizard info to register (RocketChat#14884)

* [NEW] Webdav File Picker (RocketChat#14879)

* [FIX] Invite users auto complete cropping results (RocketChat#15020)

* [FIX] Edit permissions screen (RocketChat#14950)

* fix layout still missing js

* Change publication to REST call

* Add default fields to exclude

* Use fixed fields

* Deprecate publication instead of removing it

* ui fix

* fix review

* Deprecate publication instead of removing it

* Prevent adding invalid users to role

* New: Apps and integrations statistics (RocketChat#14878)

* Add Apps and integrations infos to statistics

* Add stats for integrations with script enabled

* [FIX] Livechat dashboard average and reaction time labels (RocketChat#14845)

* [FIX] Edit message with arrow up key if not last message (RocketChat#15021)

* [IMPROVE] Connectivity Services License Sync (RocketChat#15022)

* Add workspace sync to cron job to keep license, access token, and marketplace key in sync

* Always convert the sha256 password to lowercase on checking (RocketChat#14941)

* Convert the sha256 pass to lowercase always

* Change password case on save user profile as well

* [FIX] SVG uploads crashing process (RocketChat#15006)

* Update package.json with newer sharp dependency

Shall Fix RocketChat#14944 
The underlying issue seems to be in sharp versions <0.22.x

* Update package-lock

* Fix statistics error for apps on first load (RocketChat#15026)

* [NEW] Setting to configure custom authn context on SAML requests (RocketChat#14675)

* [NEW] Accept multiple redirect URIs on OAuth Apps (RocketChat#14935)

* [NEW] Settings to further customize GitLab OAuth (RocketChat#15014)

Co-Authored-By: Hajo Möller <dasjoe@gmail.com>

* Add missing French translation (RocketChat#15013)

Add Message_AllowConvertLongMessagesToAttachment translation

* [FIX] users.setStatus REST endpoint not allowing reset status message (RocketChat#14916)

* [NEW] Options to filter discussion and livechat on Admin > Rooms (RocketChat#15019)

* [FIX] Typo in german translation (RocketChat#14833)

* Wrong text when reporting a message (RocketChat#14515)

* [FIX] Message attachments not allowing float numbers (RocketChat#14412)

* Fix russian grammatical errors in i18n (RocketChat#14622)

* [NEW] Deprecate MongoDB version 3.2 (RocketChat#15025)

* [NEW] Subscription enabled marketplace (RocketChat#14948)

* Show the subscription apps different from regular purchase prices

* Update app download calls return a Buffer directly

* Add X-Apps-Engine-Version header to marketplace requests

* Add getActiveUserCount method to apps-engine user bridge

* Add distinction  for purchase type on apps list

* Fix the issues with displaying subscriptions

* Remove external federated users from active user count

* Fix usage of federation module

* Change app installation to validate license

* Change the bridges to correctly query the workspace public key

* Change marketplaceUrl to marketplace-beta

* Update Apps and Marketplace styles (temp)

* Update price column on Marketplace

* Update status column on Marketplace

* Fix Marketplace app list update

* Remove log from client orchestrator

* Refactor server orchestrator

* Change rest api for license validation

* Add popover to Marketplace app list

* Add card (subscription) icon

* Update active user count method

* Update appManage template (partial)

* Update appManage template

* Add missing i18n strings

* Add cron routine to update apps info

* Add options parameter to new methods on model Users

* Revert testing settings

* Bump Apps-Engine version

* [NEW] Show helpful error when oplog is missing (RocketChat#14954)

* Show helpful error when oplog is missing

* Remove unused files

* Bump version to 1.3.0-rc.0

* Service Account bugs fixed

* Service Account bugs fixed

* Login bug fixed

* Username exists error test for service account added

* Regression: Framework version shouldn't be applied to the request to get an app's versions (RocketChat#15039)

* Regression: fix code style, setup wizard error and profile page header (RocketChat#15041)

* Bump version to 1.3.0-rc.1

* Update Livechat widget (RocketChat#15046)

* Service Account logout fixed

* Bump version to 1.3.0-rc.2

* [FIX] Not sanitized message types (RocketChat#15054)

* Regression: Webdav File Picker search and fixed overflows (RocketChat#15027)

* fix multiple calls (RocketChat#15060)

* Regression: getSetupWizardParameters (RocketChat#15067)

* Bump version to 1.3.0-rc.3

* Regression: Apps and Marketplace UI issues (RocketChat#15045)

* Alert admins about apps on invalid state

* Implement ui for warning and error alerts in apps

* Open detail modal on viewing subscription info instead of the subscribe one.  Check license after close

* Implement ui for failed state of apps in detail screen

* Add failure alert support into appManage

* Show validation erros/warnings on app detail page

* Add status column to apps template

* Update uninstall modal

* Notify admins of disabled apps with valid licenses

* Regression: uninstall subscribed app modal (RocketChat#15077)

* Add option to change cancel button color on modal

* Avoid an extra modal when uninstalling subscribed app

* Regression: Marketplace app pricing plan description (RocketChat#15076)

Co-Authored-By: Diego Sampaio <chinello@gmail.com>

* Regression: displaying errors for apps not installed from Marketplace (RocketChat#15075)

* Regression: Improve apps bridges for HA setup (RocketChat#15080)

* Bump version to 1.3.0-rc.4

* Bump version to 1.3.0

* Bump version to 1.4.0-develop

* [BREAK] Replace tap:i18n to add support to 3-digit locales (RocketChat#15109)

* Remove tap-i18n source code

* Replace all references to new package rocketchat:tap-i18n

* Replace package francocatena:status by own template

* Add translations for status messages

Closes RocketChat#15090
Closes RocketChat#3712

* Add support to 3-digit locale

* [FIX] cachedcollection calling multiple times SYNC (RocketChat#15104)

* fixed cachedcollection calling multiple times

* fix tests

* fix review

* Improve url validation inside message object (RocketChat#15074)

* LingoHub based on develop (RocketChat#15115)

* LingoHub Update 🚀

Manual push by LingoHub User: Diego Sampaio.
Project: Rocket.Chat

Made with ❤️ by https://lingohub.com

* Add new step to build Docker image from PRs for production again (RocketChat#15124)

* Fix automated test for manual user activation (RocketChat#14978)

* Allow file upload paths on attachments URLs (RocketChat#15121)

* [FIX] Direct Message names not visible on Admin panel (RocketChat#15114)

* [FIX] Custom emoji limit table scroll (RocketChat#15119)

* [IMPROVE] Message tooltips as everyone else (RocketChat#15135)

* Regression: cachedCollection wrong callback parameters (RocketChat#15136)

* Update pt-BR.i18n.json (RocketChat#15083)

Fix wrong word.

* removed unwanted code (RocketChat#15078)

* [FIX] Remove new hidden file and fix for .env files for Snap (RocketChat#15120)

* [FIX] Prevent to create discussion with empty name (RocketChat#14507)

* [FIX] Threads contextual bar button visible even with threads disabled (RocketChat#14956)

* Update to version 2.0.0-develop (RocketChat#15142)

* Fix custom auth (RocketChat#15141)

* [NEW] Jitsi meet room access via a token (RocketChat#12259)

* closes RocketChat#7611 - append token auth in jitsi videoconference

* Ádd missing imports

* Fix imports and jws dep

* Add option to limit the token validity to the room that it was generated for

* use canAccessRoom for jitsi:generateAccessToken to prevent abuse

* fix diff in jitsi external api

* Make settings not public

* Add wreiske to catbot config (RocketChat#15147)

* [NEW] Integrate DEEPL translation service to RC core (RocketChat#12174)

* service extension for deepl

* Add translations

* Add lint-fix-command

* Adhere to new linting rules

* Hard code service enpoint URLs of providers
Having it as a setting increases complexity (user has to enter the URL by himself) without significant benefit.
If the URL changes, the code has to be adapted anyway

* Fix translation of attachment descriptions by DeepL

* Fix getSupportedLanguages by Google

* move renameSetting to Settings model

* Add migrations for rename settings

* update package.json

* Fix formatting to make consistent with the other code

* Add additional comments

* Use active provider for context menu translations

* Add invidual key for each providers

* Nothing to migrate

* Fix spelling in comment

* Replace TAPi18n library with rocketchat:tapi18n

* Use correct setting

* Remove GPG file (RocketChat#15146)

* Switch outdated roadmap to point to milestones (RocketChat#15156)

* LingoHub based on develop (RocketChat#15166)

* LingoHub Update 🚀

Manual push by LingoHub User: Diego Sampaio.
Project: Rocket.Chat

Made with ❤️ by https://lingohub.com

* Update latest Livechat widget version(1.1.3) (RocketChat#15154)

* Update the latest livechat widget version(1.1.2).

* Rename the latest release from 1.1.2 to 1.1.3

* [FIX] IE11 - callback createTreeWalker doesnt accept acceptNo… (RocketChat#15157)

* Update latest Livechat widget version to 1.1.4 (RocketChat#15173)

* [FIX] search messages scroll (RocketChat#15175)

* [FIX] Attachment download button behavior (RocketChat#15172)

* Update presence package (RocketChat#15178)

* [FIX] TabBar not loading template titles (RocketChat#15177)

* [NEW] Endpoint to fetch livechat rooms with several filters (RocketChat#15155)

* WIP: livechat endpoint to get all rooms with filters

* Created an endpoint to get all livechat rooms with filters

* Remove wrong expectation

* Move endpoint to another folder

* simplify operation

* [BREAK] remove old livechat client (RocketChat#15133)

* fix stylelint

* Broadcast rate limiter added

* Regression: remove livechat cache from circle ci (RocketChat#15183)

* remove livechat cache from circle ci

* removed eslint

* Update Livechat to 1.1.6 (RocketChat#15186)

* [BREAK] Remove publication `usersInRole` (RocketChat#15194)

* [BREAK] Remove publication `roomSubscriptionsByRole` (RocketChat#15193)

* [BREAK] Remove REST endpoint `/api/v1/info` (RocketChat#15197)

* [BREAK] Remove GraphQL and grant packages (RocketChat#15192)

* Change notifications file imports to server (RocketChat#15184)

* Service Accounts Server tests added

* Lint errors fixed

* lint errors fixed

* Add service account button location changed

* lint errors fixed

* Test error fixed

* Sidebar header condition modified

* Fixed lint errors
mrsimpson pushed a commit to assistify/Rocket.Chat that referenced this pull request Aug 27, 2019
* service extension for deepl

* Add translations

* Add lint-fix-command

* Adhere to new linting rules

* Hard code service enpoint URLs of providers
Having it as a setting increases complexity (user has to enter the URL by himself) without significant benefit.
If the URL changes, the code has to be adapted anyway

* Fix translation of attachment descriptions by DeepL

* Fix getSupportedLanguages by Google

* move renameSetting to Settings model

* Add migrations for rename settings

* update package.json

* Fix formatting to make consistent with the other code

* Add additional comments

* Use active provider for context menu translations

* Add invidual key for each providers

* Nothing to migrate

* Fix spelling in comment

* Replace TAPi18n library with rocketchat:tapi18n

* Use correct setting
@sampaiodiego sampaiodiego mentioned this pull request Sep 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants