Add pipeline for fetching content

.github/workflows/ai-assistant-fetch-content.yml

@@ -0,0 +1,74 @@
+name: Fetch content for OpenAI assistant
+
+on:
+  workflow_dispatch:
+    inputs:
+      purge-all-content:
+        type: boolean
+        description: Purge all content
+        default: false
+
+env:
+  git-user: github-actions[bot]
+  git-email: 41898282+github-actions[bot]@users.noreply.github.com
+
+jobs:
+  update-content:
+    name: Fetch content for OpenAI assistant
+    runs-on: ubuntu-22.04
+    env:
+      PURGE_ALL_CONTENT: ${{inputs.purge-all-content}}
+      BRANCH_NAME: fetch-ai-content-${{ github.run_id }}-${{ github.run_attempt }}
+    steps:
+      - name: "Checkout code"
+        uses: actions/checkout@v4
+        with:
+          token: ${{ secrets.RELEASE_GH_DEV_ACCOUNT_PAT }}
+
+      - name: "Set up Go"
+        uses: actions/setup-go@v5
+        with:
+          go-version-file: 'go.mod'
+          # When the files to be extracted are already present,
+          # tar extraction in Golangci Lint fails with the "File exists"
+          # errors. These files appear to be present because of
+          # cache in setup-go, on disabling the cache we are no more seeing
+          # such error. Cache is to be enabled once the fix is available for
+          # this issue:
+          # https://github.com/golangci/golangci-lint-action/issues/807
+          cache: false
+
+      - name: Setup Node
+        uses: volta-cli/action@v4
+
+      - name: Set git 'user.name' and 'user.email'
+        run: |
+          git config user.name "${{ env.git-user }}"
+          git config user.email ${{ env.git-email }}
+
+      - name: Create a branch branch
+        run: |
+          git checkout -b ${{ env.BRANCH_NAME }}
+
+      - name: Fetch content
+        run: |
+          cd ./hack/assistant-setup
+          npm install
+          npm run fetch-content
+
+      - name: Push changes to remote
+        run: |
+          git add .
+          git commit -m "Refetch content for OpenAI assistant"
+          git push --set-upstream origin ${{ env.BRANCH_NAME }}
+
+      - name: Create a pull request
+        env:
+          GITHUB_TOKEN: ${{ secrets.RELEASE_GH_DEV_ACCOUNT_PAT }}
+        run: |
+          gh pr create \
+            --title "Refetch content for OpenAI assistant"  \
+             --head "${{ env.BRANCH_NAME }}" \
+             --base main \
+             --label release-automation \
+             --body "This Pull Request is created by automation to update the content for OpenAI assistant."

.github/workflows/update-ai-assistant.yml → .github/workflows/ai-assistant-reconfigure.yml

@@ -1,4 +1,4 @@
-name: Update AI assistant
+name: Reconfigure OpenAI assistant
 
 on:
   push:
@@ -12,7 +12,7 @@ env:
   OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
 
 jobs:
-  update-ai-assistant:
+  reconfigure-ai-assistant:
     name: Reconfigure OpenAI assistant
     runs-on: ubuntu-22.04
     steps:

README.md

@@ -58,18 +58,27 @@ This repository shows Botkube Cloud plugins.
 
 ## Release
 
-### Fetch Botkube content and update OpenAI assistant
+### Fetch content for OpenAI assistant
 
+The AI plugin uses Botkube content (website, docs, blog posts, etc.). To refresh it, follow the steps:
+
+1. Navigate to the [` Fetch content for OpenAI assistant` GitHub Actions workflow](https://github.com/kubeshop/botkube-cloud-plugins/actions/workflows/ai-assistant-fetch-content.yml).
+1. Optionally check the "Purge all content" checkbox.
+1. Trigger the pipeline.
 
 
+The content is shared between dev and prod AI plugin, and is synchronized during the [plugin relese](#release-plugins-for-botkube-agent).
+
 ### Release plugins for Botkube Agent
 
 The latest plugins from `main` are released automatically to the bucket `botkube-cloud-plugins-latest`.
-To release a new version of the plugins:
+To release a new production version of the plugins:
 
 1. Navigate to the [`Trigger release` GitHub Actions workflow](https://github.com/kubeshop/botkube-cloud-plugins/actions/workflows/release.yml).
 1. Provide the target version in the `version` input. Do not forget about the `v` prefix.
 
    For example, if you want to release version `1.2.3`, you should provide `version: v1.2.3`.
 
 1. Trigger the pipeline.
+
+For both latest and production plugins, the OpenAI assistant for AI plugin is reconfigured automatically concurrently to the plugins build.

hack/assistant-setup/README.md

@@ -43,7 +43,7 @@ npm run fetch-content
 
 By default, before refetching content starts:
 
-- downloaded files starting with `https://botkube.io/blog/*` and `https://botkube.io/learn/*` prefixes are kept and won't be updated,
+- downloaded files starting with [package.json](package.json)`https://botkube.io/blog/*` and `https://botkube.io/learn/*` prefixes are kept and won't be updated,
 - all other content is removed.
 
 To refetch all content, run the following command: