Switch to manual Crowdin uploading and downloading

Changes highlights
==================

This commit adds two GitHub Actions workflow:

i18n-pull
---------

i18n-pull downloads translations from Crowdin. It is run automatically
every Monday at 7 AM GMT time. It builds the Crowdin project and then
downloads all the files corresponding to the branch it is running for
(for scheduled runs it is the default branch of the repository).

i18n-push
---------

i18n-push runs the the extractor and uploads the source string files to
Crowdin for translation. It is running for every push to the `master`
branch and checks the changed files, trying to avoid unnecessary runs
when none of the files changed contain translations.

It can also be dispatched manually for every branch of the repository,
which allows to create feature branches on Crowdin and give translators
an early start. Translations for such branches can be downloaded by
dispatching i18n-pull. They will also migrate automatically if the
feature branch gets merged (see below).

i18n-cleanup
------------

i18n-cleanup performs a cleanup of every repository branch deletion by
deleting the translation branch on Crowdin as well Git pull branch.
Deleting the translation branch after the merge allows Crowdin to
migrate the translations to the other branch where updated strings now
match (if no strings match, e.g., after PR rejection, then translations
will just be deleted). Deletion of the pull branch closes the associated
pull request which is most likely no longer relevant.

***

Since translations are now extracted automatically, there is no more
need in index.json file being a part of the repository, which previously
caused pains when dealing with pull requests, so it is removed now.
Contibutors will have to run `pnpm intl:extract` manually now.

Migration strategy
==================

Please take extra care when migrating to this approach to avoid the loss
of translations.

1 - Creating a Crowdin token
----------------------------

The first step is to create a token in order for Crowdin CLI to
perform operations.

1. Navigate to the settings for the automated account on Crowdin, and
   then switch to API tab. Alternatively, use the link:
   https://crowdin.com/settings#api-key.

2. Click on "New Token"

3. Give token a meaningful name to distinguish its requests in security
   logs

4. In scopes select the following:
   - Projects
     - Projects (List, Get, Create, Edit)
     - Source files & strings
     - Translations

5. Click "Create"

6. Confirm your identity by signing in again

7. Save the access token somewhere temporarily, you will need it later

2 - Creating a GitHub token
---------------------------

In order for the download workflow to be able to create proper pull
requests, it needs to puppeteer a user account. This is required for
several reasons:

- Other actions do not run on PRs created by actions

- GitHub does not allow actions to create pull requests by default
  regardless the requested permissions in the workflow file. It requires
  enabling a dangerous permission in the organisation settings.

- Bot's identity. Bot's cute.

To create a token:

1. Log in to the GitHub account that will be automated

2. Go to Settings > Developer Settings > Personal access tokens
   > Tokens (classic)

3. Click "Generate new token" and select "Generate new token (classic)"
   from the menu

4. Give token a meaningful name and reasonable expiration time

5. In scopes select `repo` scope

6. Save the token

3 - Setting up GitHub Actions secrets and variables
--------------------------------------------------

For workflows to work correctly, they need several variables. Assuming
you completed the above steps, here's how you set them up.

1. Navigate to the modrinth/knossos repository

2. Switch to Settings tab

3. Go to Secrets and variables > Actions

4. Create the following repository secrets:
   - `CROWDIN_PERSONAL_TOKEN`: [your Crowdin token]
   - `CROWDIN_GH_TOKEN`: [your GitHub automated account token]

5. Create the following repository variables:
   - `CROWDIN_PROJECT_ID`: [Crowdin project ID (Modrinth - 518556)]

4 - Preparing Crowdin project
-----------------------------

Because workflows will replace GitHub integration, that integration
needs to be disabled before the merge to avoid conflics and accidental
deletion of the translations due to 'deletion' of the source files.

1. Go to Modrinth Crowdin project

2. Switch to Integrations tab

3. Click on GitHub integration

4. Click on modrinth/knossos repository

5. Click "Delete integration"

6. Confirm deletion

5 - Merge
---------

After following all the steps above you should be now ready to merge
this pull request.

Make sure to check that actions do run after you merge it.

.github/workflows/i18n-cleanup.yml

@@ -0,0 +1,103 @@
+name: Crowdin (cleanup)
+
+on: [delete]
+
+concurrency:
+  group: i18n-management
+
+env:
+  CROWDIN_CLI_VERSION: '3.18.0'
+
+jobs:
+  preflight_check:
+    name: 'Pre-flight check'
+    runs-on: ubuntu-22.04
+    concurrency:
+      group: i18n-cleanup:${{ github.event.ref }}
+      cancel-in-progress: true
+    if: github.event.ref_type == 'branch' && !startsWith(github.event.ref, 'ref/heads/crowdin-pull/')
+    steps:
+      - name: Preflight check
+        id: check
+        shell: bash
+        run: |
+          PREFLIGHT_CHECK_RESULT=true
+
+          function flight_failure () {
+            if [ "$PREFLIGHT_CHECK_RESULT" = true ]; then
+              echo "One or more pre-flight checks failed!"
+              echo ""
+              PREFLIGHT_CHECK_RESULT=false
+            fi
+            echo "- $1"
+          }
+
+          if [ "$CROWDIN_PROJECT_ID_DEFINED" != true ]; then
+            flight_failure "CROWDIN_PROJECT_ID variable is not defined (required to push)"
+          fi
+
+          if [ "$CROWDIN_PERSONAL_TOKEN_DEFINED" != true ]; then
+            flight_failure "CROWDIN_PERSONAL_TOKEN secret is not defined (required to push)"
+          fi
+
+          echo "flight_ok=$PREFLIGHT_CHECK_RESULT" >> "$GITHUB_OUTPUT"
+        env:
+          CROWDIN_PROJECT_ID_DEFINED: ${{ vars.CROWDIN_PROJECT_ID != '' }}
+          CROWDIN_PERSONAL_TOKEN_DEFINED: ${{ secrets.CROWDIN_PERSONAL_TOKEN != '' }}
+    outputs:
+      ready: ${{ steps.check.outputs.flight_ok == 'true' }}
+  delete_crowdin_branch:
+    name: 'Delete Crowdin branch'
+    runs-on: ubuntu-22.04
+    needs: preflight_check
+    if: needs.preflight_check.outputs.ready == 'true'
+    concurrency:
+      group: i18n-cleanup:${{ github.event.ref }}
+      cancel-in-progress: true
+    permissions:
+      contents: write
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Get branch name from ref
+        id: branch-name
+        shell: bash
+        run: echo "safe_branch_name=$(echo "${{ github.event.ref }}" | sed -e "s/[\\\\/\\:*?\"<>|]/_/g")" >> "$GITHUB_OUTPUT"
+
+      # Crowdin GitHub Action is currently broken and doesn't properly run commands that contain spaces
+      # See https://github.com/crowdin/github-action/issues/192
+      - name: Download Crowdin CLI
+        uses: robinraju/release-downloader@v1.9
+        with:
+          repository: crowdin/crowdin-cli
+          tag: ${{ env.CROWDIN_CLI_VERSION }}
+          fileName: 'crowdin-cli.zip'
+          out-file-path: '.crowdin'
+
+      - name: Delete translations branch on Crowdin
+        shell: bash
+        run: |
+          CROWDIN_DIR="$PWD/.crowdin"
+          unzip "$CROWDIN_DIR/crowdin-cli.zip" -d "$CROWDIN_DIR"
+          CROWDIN_CLI_JAR="$CROWDIN_DIR/${{ env.CROWDIN_CLI_VERSION }}/crowdin-cli.jar"
+          java -jar "$CROWDIN_CLI_JAR" branch delete "[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}"
+        continue-on-error: true
+        env:
+          CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
+          CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
+
+      # - name: Delete translations branch on Crowdin
+      #   uses: crowdin/github-action@v1
+      #   with:
+      #     command: 'branch'
+      #     command_args: 'delete "[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}"'
+      #   env:
+      #     CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
+      #     CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
+      #   continue-on-error: true
+
+      - name: Delete Git branch
+        shell: bash
+        run: 'git push origin :crowdin-pull/${{ github.event.ref }}'
+        continue-on-error: true

.github/workflows/i18n-pull.yml

@@ -0,0 +1,123 @@
+name: Crowdin (pull)
+
+on:
+  schedule:
+    - cron: '0 7 * * MON' # every monday at 7 am
+  workflow_dispatch: {}
+
+concurrency:
+  group: i18n-management
+
+jobs:
+  preflight_check:
+    name: 'Pre-flight check'
+    runs-on: ubuntu-22.04
+    concurrency:
+      group: i18n-pull:${{ github.ref }}
+      cancel-in-progress: true
+    steps:
+      - name: Preflight check
+        id: check
+        run: |
+          PREFLIGHT_CHECK_RESULT=true
+
+          function flight_failure () {
+            if [ "$PREFLIGHT_CHECK_RESULT" = true ]; then
+              echo "One or more pre-flight checks failed!"
+              echo ""
+              PREFLIGHT_CHECK_RESULT=false
+            fi
+            echo "- $1"
+          }
+
+          if [ "$CROWDIN_PROJECT_ID_DEFINED" != true ]; then
+            flight_failure "CROWDIN_PROJECT_ID variable is not defined (required to push)"
+          fi
+
+          if [ "$CROWDIN_PERSONAL_TOKEN_DEFINED" != true ]; then
+            flight_failure "CROWDIN_PERSONAL_TOKEN secret is not defined (required to push)"
+          fi
+
+          if [ "$CROWDIN_GH_TOKEN_DEFINED" != true ]; then
+            flight_failure "CROWDIN_GH_TOKEN secret is not defined (required to make pull requests)"
+          fi
+
+          echo "flight_ok=$PREFLIGHT_CHECK_RESULT" >> "$GITHUB_OUTPUT"
+        env:
+          CROWDIN_PROJECT_ID_DEFINED: ${{ vars.CROWDIN_PROJECT_ID != '' }}
+          CROWDIN_PERSONAL_TOKEN_DEFINED: ${{ secrets.CROWDIN_PERSONAL_TOKEN != '' }}
+          CROWDIN_GH_TOKEN_DEFINED: ${{ secrets.CROWDIN_GH_TOKEN != '' }}
+    outputs:
+      ready: ${{ steps.check.outputs.flight_ok == 'true' }}
+
+  pull_translations:
+    name: 'Pull translations from Crowdin'
+    needs: preflight_check
+    if: needs.preflight_check.outputs.ready == 'true'
+    runs-on: ubuntu-22.04
+    concurrency:
+      group: i18n-pull:${{ github.ref }}
+      cancel-in-progress: true
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+          token: ${{ secrets.CROWDIN_GH_TOKEN }}
+
+      - name: Configure Git author
+        id: git-author
+        uses: MarcoIeni/git-config@v0.1
+        env:
+          GITHUB_TOKEN: ${{ secrets.CROWDIN_GH_TOKEN }}
+
+      # Because --all flag of Crowdin CLI is currently broken we need to create a fake source file
+      # so that the CLI won't omit translations for it. See https://github.com/crowdin/crowdin-cli/issues/724
+      - name: Write fake sources
+        shell: bash
+        run: echo "{}" > locales/en-US/index.json
+
+      - name: Query branch name
+        id: branch-name
+        shell: bash
+        run: |
+          BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
+          SAFE_BRANCH_NAME=$(echo "$BRANCH_NAME" | sed -e "s/[\\\\/\\:*?\"<>|]/_/g")
+          echo "Branch name is $BRANCH_NAME (escaped as $SAFE_BRANCH_NAME)"
+          echo "branch_name=$BRANCH_NAME" >> "$GITHUB_OUTPUT"
+          echo "safe_branch_name=$SAFE_BRANCH_NAME" >> "$GITHUB_OUTPUT"
+
+      - name: Download translations from Crowdin
+        uses: crowdin/github-action@v1
+        with:
+          upload_sources: false
+          upload_translations: false
+          download_translations: true
+          push_translations: false
+          create_pull_request: false
+          crowdin_branch_name: '[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}'
+        env:
+          CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
+          CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
+
+      - name: Fix broken permissions
+        shell: bash
+        run: sudo chown -R $USER:$USER locales
+
+      - name: Create Pull Request
+        uses: peter-evans/create-pull-request@v6
+        with:
+          title: 'New translations from Crowdin (${{ steps.branch-name.outputs.branch_name }})'
+          body: |-
+            Here is your automated pull request with updated translations from [our Crowdin](https://crowdin.com/project/modrinth) for the `${{ steps.branch-name.outputs.branch_name }}` branch. Merge whenever you ready.
+
+            This pull request is created according to the `.github/workflows/i18n-pull.yml` file.
+
+            Want to update this pull request? [Dispatch this workflow again](https://github.com/${{ github.repository }}/actions/workflows/i18n-pull.yml).
+          commit-message: 'New translations from Crowdin (${{ steps.branch-name.outputs.branch_name }})'
+          branch: crowdin-pull/${{ steps.branch-name.outputs.branch_name }}
+          add-paths: locales
+          author: '${{ steps.git-author.outputs.name }} <${{ steps.git-author.outputs.email }}>'
+          committer: '${{ steps.git-author.outputs.name }} <${{ steps.git-author.outputs.email }}>'
+          token: ${{ secrets.CROWDIN_GH_TOKEN }}
+          git-token: ${{ secrets.CROWDIN_GH_TOKEN }}

.github/workflows/i18n-push.yml

@@ -0,0 +1,149 @@
+name: Crowdin (push)
+
+on:
+  push: {}
+  workflow_dispatch: {}
+
+concurrency:
+  group: i18n-management
+
+jobs:
+  preflight_check:
+    name: 'Pre-flight check'
+    runs-on: ubuntu-22.04
+    concurrency:
+      group: i18n-push:${{ github.ref }}
+      cancel-in-progress: true
+    steps:
+      - name: Preflight check
+        id: check
+        run: |
+          PREFLIGHT_CHECK_RESULT=true
+
+          function flight_failure () {
+            if [ "$PREFLIGHT_CHECK_RESULT" = true ]; then
+              echo "One or more pre-flight checks failed!"
+              echo ""
+              PREFLIGHT_CHECK_RESULT=false
+            fi
+            echo "- $1"
+          }
+
+          if [ "$CROWDIN_PROJECT_ID_DEFINED" != true ]; then
+            flight_failure "CROWDIN_PROJECT_ID variable is not defined (required to push)"
+          fi
+
+          if [ "$CROWDIN_PERSONAL_TOKEN_DEFINED" != true ]; then
+            flight_failure "CROWDIN_PERSONAL_TOKEN secret is not defined (required to push)"
+          fi
+
+          echo "flight_ok=$PREFLIGHT_CHECK_RESULT" >> "$GITHUB_OUTPUT"
+        env:
+          CROWDIN_PROJECT_ID_DEFINED: ${{ vars.CROWDIN_PROJECT_ID != '' }}
+          CROWDIN_PERSONAL_TOKEN_DEFINED: ${{ secrets.CROWDIN_PERSONAL_TOKEN != '' }}
+
+      - name: Check that Crowdin branch exists
+        if: github.event_name != 'workflow_dispatch' && github.ref_name != github.event.repository.default_branch
+        id: crowdin-branch-exists
+        uses: GuillaumeFalourd/branch-exists@v1
+        with:
+          branch: ${{ format('crowdin-pull/{0}', github.ref_name) }}
+
+      - name: Checkout
+        id: checkout
+        if: github.event_name != 'workflow_dispatch' && steps.check.outputs.flight_ok == 'true' && (steps.crowdin-branch-exists.outcome == 'skipped' || steps.crowdin-branch-exists.outputs.exists == 'true')
+        uses: actions/checkout@v4
+
+      - name: Confirm push necessity
+        id: changed-files
+        if: steps.checkout.outcome != 'skipped'
+        uses: tj-actions/changed-files@v42
+        with:
+          negation_patterns_first: true
+          files: |
+            **
+            locales/en-US/**
+            crowdin.yml
+          files_ignore: |
+            .{github,vscode,idea}/**
+            {assets,patches,types,public,locales}/**
+            .{editorconfig,gitignore,npmrc,prettierignore}
+            .*.{js,json}
+            *.{md,yml,yaml,json}
+            LICENSE
+
+      - name: Output result
+        if: steps.changed-files.outcome != 'skipped'
+        run: |
+          cat << EOF
+            Changed files are ${{ steps.changed-files.outputs.all_changed_files }}
+          EOF
+    outputs:
+      ready: ${{ steps.check.outputs.flight_ok == 'true' && (github.event_name == 'workflow_dispatch' || steps.changed-files.outputs.any_changed == 'true') }}
+
+  push_translations:
+    name: Push sources to Crowdin
+    needs: preflight_check
+    if: needs.preflight_check.outputs.ready == 'true'
+    concurrency:
+      group: i18n-push:${{ github.ref }}
+      cancel-in-progress: true
+    runs-on: ubuntu-22.04
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+
+      - name: Use Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 18.x
+
+      - name: Install pnpm via corepack
+        shell: bash
+        run: |
+          corepack enable
+          corepack prepare --activate
+
+      - name: Get pnpm store directory
+        id: pnpm-cache
+        shell: bash
+        run: echo "STORE_PATH=$(pnpm store path)" >> $GITHUB_OUTPUT
+
+      - name: Setup pnpm cache
+        uses: actions/cache@v4
+        with:
+          path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
+          key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
+          restore-keys: |
+            ${{ runner.os }}-pnpm-store-
+
+      - name: Install dependencies
+        run: pnpm install
+
+      - name: Extract translations
+        run: pnpm intl:extract
+
+      - name: Query branch name
+        id: branch-name
+        shell: bash
+        run: |
+          BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
+          SAFE_BRANCH_NAME=$(echo "$BRANCH_NAME" | sed -e "s/[\\\\/\\:*?\"<>|]/_/g")
+          echo "Branch name is $BRANCH_NAME (escaped as $SAFE_BRANCH_NAME)"
+          echo "branch_name=$BRANCH_NAME" >> "$GITHUB_OUTPUT"
+          echo "safe_branch_name=$SAFE_BRANCH_NAME" >> "$GITHUB_OUTPUT"
+
+      - name: Upload translations to Crowdin
+        uses: crowdin/github-action@v1
+        with:
+          upload_sources: true
+          upload_translations: false
+          download_translations: false
+          push_translations: false
+          create_pull_request: false
+          crowdin_branch_name: '[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}'
+        env:
+          CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
+          CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}

.gitignore

@@ -83,3 +83,6 @@ sw.*
 
 # Vim swap files
 *.swp
+
+# Automatically generated locale files
+/locales/en-US/index.json

crowdin.yml

@@ -1,8 +1,11 @@
-project_id: 518556
 preserve_hierarchy: true
 commit_message: '[ci skip]'
 
 files:
-  - source: /locales/en-US/*
+  - source: /locales/en-US/*.json
     dest: /%original_file_name%
     translation: /locales/%locale%/%original_file_name%
+    skip_untranslated_strings: true
+
+project_id_env: CROWDIN_PROJECT_ID
+api_token_env: CROWDIN_PERSONAL_TOKEN