feat(i18n): Setup react-i18next #9797

lidimayra · 2022-04-01T12:51:27Z

Setup i18n on Force using react-i18next.

For questions/suggestions, join #hack9-localizations slack channel
For context and usage, see docs available at https://github.com/artsy/force/blob/main/docs/i18n.md

lidimayra · 2022-04-01T12:54:52Z

src/v2/Components/Search/Suggestions/SuggestionItem.tsx

-export const PLACEHOLDER = "Search by artist, gallery, style, theme, tag, etc."
-export const PLACEHOLDER_XS = "Search Artsy"


These weren't being used anywhere besides inside the SearchBar component, so we deleted it from here.

lidimayra · 2022-04-01T12:59:03Z

src/v2/System/i18n/i18n.ts

+    debug: true,
+    // backend: {
+    //   loadPath: "assets/locales/{{lng}}/{{ns}}.json",
+    // },


Ideally, we would like the JSON files to be loaded like this, but we couldn't make it work (serving the static assets using webpacker... any ideas?).

we tried a few options but none worked, basically, we need to serve the locale JSON files somewhere, we tried under /assets but got lost in the setup of how to do it, any suggestions? cc @damassi 😬

Happy to pair on this when y'all have a moment to pick this back up!

Thanks @damassi, that would be great! When would be a good moment for you?

Also: is there something that should be prepared beforehand?

I'm back from OOO; please reach out in slack!

lidimayra · 2022-04-01T13:24:49Z

src/desktop/components/react/stitch_components/ReactionTooltipQuestion.tsx

@@ -1,5 +1,6 @@
 import { HelpIcon } from "@artsy/palette"
 import { Tooltip } from "v2/Components/Tooltip"
+import "v2/System/i18n/i18n"


We actually wanted that this import would happen only on the Boot component and it shouldn't be needed anywhere else. However, the components inside the stitch_components directory are used in the .jade files for rendering some of our static pages (like /terms, for example), which don't rely on Boot - this is why we need to replicate this import in some of these files.

See also this comment

I wouldn't worry about anything that exists inside of /desktop; this folder is soon going to be deleted completely.

hmm, but it's still being used right? The issue here is that if we don't make this change here, some of our existing pages crash...

Oh i see! ok, np 👍

Once we delete desktop this will just go away safely

src/v2/Components/NavBar/NavBar.tsx

This setting was leading our json files to be misinterpreted by the transpiler (build would fail trying to read json files as js files)

pvinis · 2022-07-18T14:07:47Z

jest.config.v2.js

@@ -19,7 +19,7 @@ module.exports = {
      testURL: "http://localhost",
      transform: {
        "\\.(gql|graphql)$": "jest-transform-graphql",
-        ".(ts|tsx|js|jsx)": "babel-jest",
+        "^.+\\.(ts|tsx|js|jsx)$": "babel-jest",


this regex feels weird.. the $ at the end makes sense, and should be enough to exclude the json files. the beginning part seems unnecessary?

Applied here: b5a4584

pvinis · 2022-07-18T14:08:53Z

src/v2/System/i18n/i18n.ts

+const translations = require("v2/System/locales/en-US/translation.json")
+
+i18n.use(initReactI18next).init({
+  debug: true,


do we need this true? 🤔

I thought it would be better to keep it true as it is helpful in case we have issues when translations can't be loaded (default value is false). Would it be better to remove it?

as long as it doesnt break anything for real user on prod, then lets keep it here. I'm just not sure what this true entails, so I was being cautious.

Good call!
This is the official definition: https://www.i18next.com/overview/configuration-options

logs info level to console output. Helps finding issues with loading not working

As I understand, it's just about logs, so I don't think it can break anything. Do you think I could be missing something?

makes sense. all good then :)

We should only enable this in development, which is as easy as checking debug: getENV("NODE_ENV") === "development"

pvinis · 2022-07-18T14:09:38Z

src/v2/Components/NavBar/NavBar.tsx

@@ -149,7 +151,7 @@ export const NavBar: React.FC = track(
                  flex={1}
                  size="small"
                >
-                  Sign Up
+                  {t`navbar.signup`}


pvinis · 2022-07-18T14:12:32Z

src/v2/Components/NavBar/__tests__/NavBar.jest.tsx

@@ -62,17 +62,17 @@ describe("NavBar", () => {
  describe("desktop", () => {
    it("renders logged out items", () => {
      const wrapper = getWrapper()
-      expect(wrapper.html()).toContain("Log In")
-      expect(wrapper.html()).toContain("Sign Up")
+      expect(wrapper.html()).toContain("navbar.login")


this is strange.. should it contain the key? feels like the test should be testing if the actual text is there.

something like

it("works in English", () => { expect(getWrapper().html()).toContain("Log In") }) it("works in Greek", () => { someFuncToMockBrowserLang("gr") expect(getWrapper().html()).toContain("Σύνδεση") })

Yeah, this is something that we also discussed during the implementation... but if the goal here is that at some point the language experts should be capable of using a third-party service to update the translations as they wish, do we want the tests to fail when the content of text changes?

i would say yes 🤔.

i guess we should support both? it would be great to be able to test the actual text we expand to, to test that languages are actually applied correctly, but also we should allow for key strings to be checked, though that can just be the testID 🤔. Im not set on one or the other.

Maybe we start with something and see how it goes. I just feel like checking just for the key might take us to a place where a translation is missing, and we never know, because our test is green, and our website says navbar.signup instead of Sign Up.

Yeah, this is a super valid point
I added 1b51258 to change the test to look for the strings instead of keys

pvinis · 2022-07-18T14:14:47Z

src/v2/Components/Search/SearchBar.tsx


 const logger = createLogger("Components/Search/SearchBar")

-export interface Props extends SystemContextProps {
+export interface Props extends SystemContextProps, WithTranslation {


why would we do this here this way, and not with the same useTranslation hook?

oh i see.. this is a class component.

even though it makes sense, I would much prefer it if we used something like https://github.com/artsy/eigen/blob/26a98c408c223efefbb0b8309448c3bb79256ab3/src/app/Scenes/Search/components/CityGuideCTA.tsx#L12-L13 (docs here at the bottom https://github.com/artsy/eigen/blob/b4484db7fb7ddfbad4e903c993628bbd90a909b3/docs/palette_v3_migration_guide.md)
so we can keep everything nice and centralized, with a t exposed.

Thanks @pvinis !! ❤️

As this is not part of an actual project, I don't think I'll have time to focus a lot on these changes before Wednesday. Do you think it would make sense to remove the SearchBar translations from this PR so we could make it smaller by applying the i18n to the NavBar only? And then working on the SearchBar on the next Future Friday?

If you think the way I suggest is a good way to do this change for the file here, I'm happy to help or even make a PR on top of this one with the change. If you think we need more discussion on this, I'll gladly greenlight this 👍.

I completely agree that your suggestion would be much cleaner, if it's possible to open a PR on top of this one, that would be great. I just don't want to put any pressure on you either, I'm already super grateful for the careful code review. ❤️

I'll take a quick stab at it.

pvinis

some questions and some suggestions 🤔

pvinis

approving this, since the comments are non-blocking. happy to pair or make a PR on top of this for a <ClassI18n> component to simplify the class comp case.

* simplifying * chiller types

lidimayra · 2022-07-18T16:50:47Z

Thank you so much for the PR, @pvinis, I'm merging this now ❤️

src/v2/System/i18n/ClassI18n.tsx

artsy-peril · 2022-07-18T18:29:22Z

react-i18next

Author: Jan Mühlemann

Description: Internationalization for react done right. Using the i18next i18n ecosystem.

Homepage: https://github.com/i18next/react-i18next


Created	over 6 years ago
Last Updated	12 days ago
License	MIT
Maintainers	2
Releases	264
Direct Dependencies	`@babel/runtime` and `html-parse-stringify`
Keywords	i18next, internationalization, i18n, translation, localization, l10n, globalization, react and reactjs

This README is too long to show.

insane

Author: Nicolas Bevacqua

Description: Lean and configurable whitelist-oriented HTML sanitizer

Homepage: https://github.com/bevacqua/insane


Created	over 7 years ago
Last Updated	about 1 month ago
License	MIT
Maintainers	1
Releases	18
Direct Dependencies	`assignment` and `he`

README

insane

Lean and configurable whitelist-oriented HTML sanitizer

Works well in browsers, as its footprint size is very small (around ~2kb gzipped). API inspired by sanitize-html (which is around 100kb gzipped).

_{You would be insane not to use this!}

Install

npm install insane --save

Usage

insane('<div>foo<span>bar</span></div>', { allowedTags: ['div'] })
// <- '<div>foo</div>'

Contrary to similar sanitizers, insane drops the whole tree of descendants for elements that aren't allowed tags.

API

`insane(html, options?, strict?)`

html can be an arbitrary HTML string
options are detailed below
strict means that options won't be based off of insane.defaults if set to true

The parser takes into account that some elements can be self-closing. For safety reasons the sanitizer will only accept a valid URL for background, base, cite, href, longdesc, src, and usemap elements. "Valid URL" means that it begins with either #, /, or any of options.allowedSchemes (followed by :).

`options`

Sensible defaults are provided. You can override specific options as needed.

`allowedSchemes`

Defaults to ['http', 'https', 'mailto'].

`allowedTags`

An array of tags that you'll allow in the resulting HTML.

Example

Only allow spans, discarding the rest of elements.

insane('<div>foo</div><span>bar</span>', {
  allowedTags: ['span']
});
// <- '<span>bar</span>'

`allowedAttributes`

An object describing the attributes you'll allow for each individual tag name.

Example

Only allow spans, and only allow those spans to have an id (discarding the rest of their attributes).

insane('<span id="bar" class="super">bar</span>', {
  allowedTags: ['span'],
  allowedAttributes: { span: ['id'] }
});
// <- '<span id="bar">bar</span>'

`allowedClasses`

If 'class' is listed as an allowed attribute, every single class will be allowed. If you don't list 'class' as an allowed attribute, you can provide a class whitelist per tag name.

Example

Only allow spans to have super or bad class names, discarding the rest of them.

insane('<span class="super mean and bad">bar</span>', {
  allowedTags: ['span'],
  allowedClasses: { span: ['super', 'bad'] }
});
// <- '<span class="super bad">bar</span>'

`filter`

Takes a function(token) that allows you to do additional validation beyond exact tag name and attribute matching. The token object passed to your filter contains the following properties.

tag is the lowercase tag name of the element
attrs is an object containing every attribute in the element, including those that may not be in the whitelist

If you return a falsy value the element and all of its descendants will not be included in the output. Note that you are allowed to change the attrs, and even add new ones, transforming the output.

Example

Require that <span> elements have an aria-label value.

function filter (token) {
  return token.tag !== 'span' || token.attrs['aria-label'];
}
insane('<span aria-label="a foo">foo</span><span>bar</span>', {
  allowedTags: ['span'],
  allowedAttributes: { span: ['aria-label'] },
  filter: filter
});
// <- '<span aria-label="a foo">foo</span>'

`transformText`

Takes a function(text) that allows you to modify text content in HTML elements. Runs for every piece of text content. The returned value is used instead of the original text contents.

Defaults

The default configuration is used if you don't provide any. This object is available at insane.defaults. You are free to manipulate the defaults themselves.

{
  "allowedAttributes": {
    "a": ["href", "name", "target"],
    "iframe": ["allowfullscreen", "frameborder", "src"],
    "img": ["src"]
  },
  "allowedClasses": {},
  "allowedSchemes": ["http", "https", "mailto"],
  "allowedTags": [
    "a", "article", "b", "blockquote", "br", "caption", "code", "del", "details", "div", "em",
    "h1", "h2", "h3", "h4", "h5", "h6", "hr", "i", "img", "ins", "kbd", "li", "main", "ol",
    "p", "pre", "section", "span", "strike", "strong", "sub", "summary", "sup", "table",
    "tbody", "td", "th", "thead", "tr", "u", "ul"
  ],
  "filter": null,
  "transformText": null
}

License

MIT

i18next-browser-languagedetector

Author: Jan Mühlemann

Description: language detector used in browser environment for i18next

Homepage: https://github.com/i18next/i18next-browser-languageDetector


Created	over 6 years ago
Last Updated	about 1 month ago
License	MIT
Maintainers	2
Releases	49
Direct Dependencies	`@babel/runtime`
Keywords	i18next and i18next-languageDetector

README

Introduction

This is a i18next language detection plugin use to detect user language in the browser with support for:

cookie (set cookie i18next=LANGUAGE)
sessionStorage (set key i18nextLng=LANGUAGE)
localStorage (set key i18nextLng=LANGUAGE)
navigator (set browser language)
querystring (append ?lng=LANGUAGE to URL)
htmlTag (add html language tag <html lang="LANGUAGE" ...)
path (http://my.site.com/LANGUAGE/...)
subdomain (http://LANGUAGE.site.com/...)

Getting started

Source can be loaded via npm, bower or downloaded from this repo.

# npm package
$ npm install i18next-browser-languagedetector

# bower
$ bower install i18next-browser-languagedetector

If you don't use a module loader it will be added to window.i18nextBrowserLanguageDetector

Wiring up:

import i18next from 'i18next';
import LanguageDetector from 'i18next-browser-languagedetector';

i18next.use(LanguageDetector).init(i18nextOptions);

As with all modules you can either pass the constructor function (class) to the i18next.use or a concrete instance.

Detector Options

The default options can be found here.

{
  // order and from where user language should be detected
  order: ['querystring', 'cookie', 'localStorage', 'sessionStorage', 'navigator', 'htmlTag', 'path', 'subdomain'],

  // keys or params to lookup language from
  lookupQuerystring: 'lng',
  lookupCookie: 'i18next',
  lookupLocalStorage: 'i18nextLng',
  lookupSessionStorage: 'i18nextLng',
  lookupFromPathIndex: 0,
  lookupFromSubdomainIndex: 0,

  // cache user language on
  caches: ['localStorage', 'cookie'],
  excludeCacheFor: ['cimode'], // languages to not persist (cookie, localStorage)

  // optional expire and domain for set cookie
  cookieMinutes: 10,
  cookieDomain: 'myDomain',

  // optional htmlTag with lang attribute, the default is:
  htmlTag: document.documentElement,

  // optional set cookie options, reference:[MDN Set-Cookie docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie)
  cookieOptions: { path: '/', sameSite: 'strict' }
}

Options can be passed in:

preferred - by setting options.detection in i18next.init:

import i18next from 'i18next';
import LanguageDetector from 'i18next-browser-languagedetector';

i18next.use(LanguageDetector).init({
  detection: options,
});

on construction:

import LanguageDetector from 'i18next-browser-languagedetector';
const languageDetector = new LanguageDetector(null, options);

via calling init:

import LanguageDetector from 'i18next-browser-languagedetector';
const languageDetector = new LanguageDetector();
languageDetector.init(options);

Adding own detection functionality

interface

export default {
  name: 'myDetectorsName',

  lookup(options) {
    // options -> are passed in options
    return 'en';
  },

  cacheUserLanguage(lng, options) {
    // options -> are passed in options
    // lng -> current language, will be called after init and on changeLanguage
    // store it
  },
};

adding it

import LanguageDetector from 'i18next-browser-languagedetector';
const languageDetector = new LanguageDetector();
languageDetector.addDetector(myDetector);

i18next.use(languageDetector).init({
  detection: options,
});

Don't forget: You have to add the name of your detector (myDetectorsName in this case) to the order array in your options object. Without that, your detector won't be used. See the Detector Options section for more.

Gold Sponsors

localization as a service - locize.com

Needing a translation management? Want to edit your translations with an InContext Editor? Use the orginal provided to you by the maintainers of i18next!

With using locize you directly support the future of i18next and react-i18next.

i18next

Author: Jan Mühlemann

Description: i18next internationalization framework

Homepage: https://www.i18next.com


Created	over 10 years ago
Last Updated	12 days ago
License	MIT
Maintainers	2
Releases	381
Direct Dependencies	`@babel/runtime`
Keywords	i18next, internationalization, i18n, translation, localization, l10n, globalization and gettext

README

i18next: learn once - translate everywhere

i18next is a very popular internationalization framework for browser or any other javascript environment (eg. Node.js, Deno).

i18next provides:

Flexible connection to backend (loading translations via xhr, ...)
Optional caching, user language detection, ...
Proper pluralizations
Translation context
Nesting, Variable replacement
Flexibility: Use it everywhere
Extensibility: eg. sprintf
...

For more information visit the website:

Our focus is providing the core to building a booming ecosystem. Independent of the building blocks you choose, be it react, angular or even good old jquery proper translation capabilities are just one step away.

Documentation

The general i18next documentation is published on www.i18next.com and PR changes can be supplied here.

The react specific documentation is published on react.i18next.com and PR changes can be supplied here.

Gold Sponsors

From the creators of i18next: localization as a service - locize.com

A translation management system built around the i18next ecosystem - locize.com.

With using locize you directly support the future of i18next.

New dependencies added: i18next, i18next-browser-languagedetector, insane and react-i18next.

Generated by 🚫 dangerJS against 501ee36

damassi · 2022-07-18T18:30:00Z

🎉

src/v2/System/locales/en/translation.json

We setup react-i18next during [these changes](#9797), these docs contain instructions about the context, usage and next steps. Co-authored-by: Lidiane Taquehara <lidi.mayra@gmail.com> Co-authored-by: Leandro Linares <leandro.linares@artsymail.com>

* Include docs about i18n We setup react-i18next during [these changes](#9797), these docs contain instructions about the context, usage and next steps. Co-authored-by: Lidiane Taquehara <lidi.mayra@gmail.com> Co-authored-by: Leandro Linares <leandro.linares@artsymail.com> * Move locales to /i18n directory * Add comment about assets served on the server-side * camelCase instead of lowerCamelCase * Update docs/i18n.md Co-authored-by: Pavlos Vinieratos <pvinis@gmail.com> Co-authored-by: Leandro Linares <leandro.linares@artsymail.com> Co-authored-by: Pavlos Vinieratos <pvinis@gmail.com>

lidimayra added the hackathon label Apr 1, 2022

lidimayra requested review from jonallured, dzucconi and damassi April 1, 2022 12:51

lidimayra assigned araujobarret Apr 1, 2022

lidimayra requested a review from spkng-slnc April 1, 2022 12:52

lidimayra commented Apr 1, 2022

View reviewed changes

lidimayra mentioned this pull request Apr 1, 2022

[FF] Setup React intl #9449

Closed

lidimayra commented Apr 1, 2022

View reviewed changes

araujobarret requested a review from pvinis April 1, 2022 13:30

lidimayra force-pushed the hack10/intl-i18next branch 2 times, most recently from d85c8f0 to fcb97f2 Compare April 1, 2022 13:42

pvinis reviewed Apr 1, 2022

View reviewed changes

src/v2/Components/NavBar/NavBar.tsx Outdated Show resolved Hide resolved

lidimayra force-pushed the hack10/intl-i18next branch from fcb97f2 to 0042ad0 Compare April 12, 2022 14:39

lidimayra force-pushed the hack10/intl-i18next branch 15 times, most recently from 264dff2 to dfd83be Compare April 29, 2022 13:51

Ensure json files are excluded in jest config

e4f424e

This setting was leading our json files to be misinterpreted by the transpiler (build would fail trying to read json files as js files)

lidimayra force-pushed the hack10/intl-i18next branch from b680848 to e4f424e Compare July 18, 2022 10:53

pvinis reviewed Jul 18, 2022

View reviewed changes

pvinis suggested changes Jul 18, 2022

View reviewed changes

Try to remove the beginning of the regex on config

b5a4584

dzucconi removed their request for review July 18, 2022 14:35

pvinis previously approved these changes Jul 18, 2022

View reviewed changes

Test actual strings intead of keys

1b51258

lidimayra dismissed pvinis’s stale review via 1b51258 July 18, 2022 15:36

simplifying (#10547)

501ee36

* simplifying * chiller types

lidimayra merged commit 15693e3 into main Jul 18, 2022

lidimayra deleted the hack10/intl-i18next branch July 18, 2022 16:51

artsy-peril bot mentioned this pull request Jul 18, 2022

Deploy #10548

Merged

damassi reviewed Jul 18, 2022

View reviewed changes

src/v2/System/i18n/ClassI18n.tsx Show resolved Hide resolved

damassi changed the title ~~Setup react-i18next~~ feat(i18n): Setup react-i18next Jul 18, 2022

anandaroop reviewed Jul 18, 2022

View reviewed changes

src/v2/System/locales/en/translation.json Show resolved Hide resolved

damassi mentioned this pull request Jul 18, 2022

chore(port): Set to 4000 to avoid OSX gotchas #10550

Merged

lidimayra mentioned this pull request Jul 22, 2022

Include docs about i18n #10588

Merged

lidimayra mentioned this pull request Oct 25, 2022

fix(i18n): globally mocks translation function #11238

Merged

		export const PLACEHOLDER = "Search by artist, gallery, style, theme, tag, etc."
		export const PLACEHOLDER_XS = "Search Artsy"

feat(i18n): Setup react-i18next #9797

feat(i18n): Setup react-i18next #9797

Conversation

lidimayra commented Apr 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lidimayra Jul 18, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damassi Jul 18, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pvinis left a comment

Choose a reason for hiding this comment

pvinis left a comment

Choose a reason for hiding this comment

lidimayra commented Jul 18, 2022

artsy-peril bot commented Jul 18, 2022 • edited Loading

insane

Install

Usage

API

insane(html, options?, strict?)

options

allowedSchemes

allowedTags

Example

allowedAttributes

Example

allowedClasses

Example

filter

Example

transformText

Defaults

License

Introduction

Getting started

Detector Options

Adding own detection functionality

interface

adding it

Gold Sponsors

i18next: learn once - translate everywhere

Documentation

Gold Sponsors

damassi commented Jul 18, 2022

lidimayra commented Apr 1, 2022 •

edited

Loading

lidimayra Jul 18, 2022 •

edited

Loading

damassi Jul 18, 2022 •

edited

Loading

artsy-peril bot commented Jul 18, 2022 •

edited

Loading

`insane(html, options?, strict?)`

`options`

`allowedSchemes`

`allowedTags`

`allowedAttributes`

`allowedClasses`

`filter`

`transformText`