AI Chat: introduce freemium model concept #21398

petemill · 2023-12-15T09:15:14Z

Resolves:

Don't allow freemium models to be selected when griffin flag isn't pointing at a freemium model (or other flag)

Testable via
--enable-features=AIChat:is_freemium_available\/false\/default_model\/chat-basic
Modify rate limiting message text for freemium models with non-premium user: "Switch to free model"
Remove concept of separate premium user default model or make it the same model (made it the same model)
Migrate pref values: chat-default to chat-basic for llama2-13b

Submitter Checklist:

I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
There is a ticket for my issue
Used Github auto-closing keywords in the PR description above
Wrote a good PR/commit description
Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
Checked the PR locally:
- npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
- npm run lint, npm run presubmit wiki, npm run gn_check, npm run tslint
Ran git rebase master (if needed)

Reviewer Checklist:

A security review is not needed, or a link to one is included in the PR description
New files have MPL-2.0 license header
Adequate test coverage exists to prevent regressions
Major classes, functions and non-trivial code blocks are well-commented
Changes in component dependencies are properly reflected in gn
Code follows the style guide
Test plan is specified in PR before merging

After-merge Checklist:

Test Plan:

brave-builds · 2023-12-15T10:16:07Z

A Storybook has been deployed to preview UI for the latest push

brave-builds · 2023-12-15T10:59:26Z

A Storybook has been deployed to preview UI for the latest push

brave-builds · 2023-12-29T22:35:30Z

A Storybook has been deployed to preview UI for the latest push

brave-builds · 2024-01-09T04:06:35Z

A Storybook has been deployed to preview UI for the latest push

brave-builds · 2024-01-09T09:39:58Z

A Storybook has been deployed to preview UI for the latest push

…ich prompts to use a basic (free) model as secondary action

If true, certain freemium models are available to non-premium users. If false, those models are premium-only.

…basic"

petemill · 2024-01-16T08:42:08Z

browser/ui/webui/settings/brave_settings_localized_strings_provider.cc

-     IDS_SETTINGS_APPEARANCE_SETTINGS_GET_MORE_THEMES},
-    {"appearanceBraveDefaultImagesOptionLabel",
-     IDS_SETTINGS_APPEARANCE_SETTINGS_BRAVE_DEFAULT_IMAGES_OPTION_LABEL},
+      {"importExtensions", IDS_SETTINGS_IMPORT_EXTENSIONS_CHECKBOX},


This formatting must have been not done for cr121 @mkarolin? npm run format is producing this - happy to keep in this PR as I hope to merge today

petemill · 2024-01-16T08:45:08Z

browser/ui/webui/settings/brave_settings_localized_strings_provider.cc

+      {"braveLeoAssistantModelSelectionLabel",
+       IDS_SETTINGS_LEO_ASSISTANT_MODEL_SELECTION_LABEL},
+      {"braveLeoModelCategory-chat", IDS_CHAT_UI_MODEL_CATEGORY_CHAT},
+      {"braveLeoModelSubtitle-chat-basic", IDS_CHAT_UI_CHAT_BASIC_SUBTITLE},


This is the only line I actually changed in this file for this PR (aside from the formatting) - key changed from "-default" to "-basic" suffix.

… maintains iOS compat

brave-builds · 2024-01-16T10:42:08Z

A Storybook has been deployed to preview UI for the latest push

nullhook · 2024-01-16T16:43:00Z

components/ai_chat/core/browser/conversation_driver.cc

@@ -132,28 +135,29 @@ ConversationDriver::~ConversationDriver() = default;
 void ConversationDriver::ChangeModel(const std::string& model_key) {
  DCHECK(!model_key.empty());
  // Check that the key exists
-  if (kAllModels.find(model_key) == kAllModels.end()) {
+  auto* new_model = GetModel(model_key);


nit: i know you like writing auto but i can't infer the type by reading the code here.

there's a general guidance on this somewhere but briefly:

write auto if type is relatively obvious

write auto if it ocassionally needs to be inferred

nullhook · 2024-01-16T16:44:28Z

components/ai_chat/core/browser/conversation_driver.cc

  InitEngine();
 }

 const mojom::Model& ConversationDriver::GetCurrentModel() {
-  return kAllModels.find(model_key_)->second;
+  auto* model = GetModel(model_key_);


this is an example of it being relatively obvious from the above line, so this works

…ength mismatch more often

github-actions · 2024-01-16T18:54:04Z

[puLL-Merge] - brave/brave-core@21398

Description

This PR contains multiple changes across different files related to ai_chat and browser settings. The motivation seems to be to enhance and refactor the AI chat models, including UI adjustments and introducing freemium models for non-premium users. There's also an emphasis on code robustness and a slight shift in how model names are handled.

Changes

Changes

browser/ui/webui/settings/brave_settings_leo_assistant_handler.cc

Refactoring BraveLeoAssistantHandler::HandleGetModels to use a direct call to ai_chat::GetAllModels() instead of manually constructing a vector of models.

browser/ui/webui/settings/brave_settings_localized_strings_provider.cc

Various localization strings are updated, but without any key changes. This is likely in support of the ui changes.

components/ai_chat/core/browser/constants.cc

Updates related to localization strings for AI chat features, including labels and descriptions for chat models and premium features.

components/ai_chat/core/browser/conversation_driver.cc

Changing ConversationDriver model initialization to reflect new feature flag values for default and premium default model keys.
Updated engine type initialization based on the model's configuration.

components/ai_chat/core/browser/engine/engine_consumer_claude.cc and engine_consumer_llama.cc

Removed the override for the model name from feature flags. The model name is now taken directly from the mojom::Model object.

components/ai_chat/core/browser/models.cc and models.h

Refactoring of AI chat models to use vector and feature flags instead of flat maps.
New methods GetAllModels and GetModel introduced for better model management.

components/ai_chat/core/common/BUILD.gn

Added a unit test target for AI chat common components.

components/ai_chat/core/common/features.cc and features.h

Introduced feature parameters to define the default AI model key for normal and premium users.
Added a feature flag for the availability of freemium models.

components/ai_chat/core/common/mojom/ai_chat.mojom

Introduced ModelAccess enum to designate a model's availability tier (basic, freemium, premium).

components/ai_chat/core/common/pref_names.cc

Default model key migration included to change from "chat-default" to "chat-basic".

components/ai_chat/resources/page/components/alerts/error_rate_limit.tsx

Removed props and direct usage of the retry button's onRetry callback. The retry is now handled by getting the page handler instance directly within the component.

components/ai_chat/resources/page/components/feature_button_menu/index.tsx

UI adjustments for feature button menu to include freemium labels and checks based on model access level.

components/ai_chat/resources/page/components/main/index.tsx

Modified logic related to premium model checking and suggestions in the main component.

components/ai_chat/resources/page/state/context.ts and data-context-provider.tsx

Context method rename from switchToDefaultModel to switchToBasicModel to reflect actual functionality.

components/ai_chat/resources/page/stories/components_panel.tsx and locale.ts

Storybook and localization updates to reflect new naming and access levels for AI chat models.

components/resources/ai_chat_ui_strings.grdp

Updates AI chat UI strings including model subtitles and prompts for switching to basic or premium models.

Security Hotspots

No specific security hotspots have been detected in these changes. The PR mainly touches UI components, localized string definitions, and internal model management for the AI chat feature. The refactoring of model access does not seem to present direct vulnerabilities, but it is worth noting that access control changes should always be double-checked to ensure they do not introduce unintended permissions or access.

bsclifton

++ string reviewers

brave-builds · 2024-01-16T19:30:21Z

A Storybook has been deployed to preview UI for the latest push

* AI Chat: introduce freemium model concept * AI Chat: Add mixtral model * AI Chat: premium's default also becomes leo-expanded * AI Chat: non-premium users on freemium models get rate limit error which prompts to use a basic (free) model as secondary action * fix format from chromium upgrade * AI Chat: introduce "is_freemium_available" boolean feature param If true, certain freemium models are available to non-premium users. If false, those models are premium-only. * AI Chat: migrate default model key pref from "chat-default" to "chat-basic" * AI Chat: reduce claude page content character length to avoid token length mismatch more often * AI Chat: model menu uses model display name instead of API name

stephendonner · 2024-01-17T21:19:18Z

Verification `PASSED` using

Brave | 1.63.124 Chromium: 121.0.6167.57 (Official Build) nightly (64-bit)
-- | --
Revision | e0864ffae4be8015ad75c2b344ab75c2eefb31a7
OS | Windows 10 Version 22H2 (Build 19045.3930)

Allow some Premium models to be used by non-Premium users brave/brave-browser#34910 - `PASSED`

`Mixtral` - `PASSED`

Steps:

installed 1.63.124
launched Brave using --env-leo=staging --env-ai-chat.bsg=dev --env-ai-chat-premium.bsg=dev --enable-logging=stderr
clicked on Leo in the sidebar
clicked on Accept and begin
loaded vox.com
clicked on ...
chose Mixtral
loaded an article
clicked on Summarize this page
clicked on Suggest questions...

example	example	example

`Claude Instant` - `PASSED`

Steps:

installed 1.63.124
launched Brave using --env-leo=staging --env-ai-chat.bsg=dev --env-ai-chat-premium.bsg=dev --enable-logging=stderr
clicked on Leo in the sidebar
clicked on Accept and begin
loaded vox.com
clicked on ...
chose Claude Instant
loaded an article
clicked on Summarize this page
clicked on Suggest questions...

example	example	example

`llama2-13b` - `PASSED`

example	example	example

Add Griffin feature flag for setting `Llama-13b` (actually `Mixtral`) as the default: brave/brave-browser#34721 - `PASSED`

Steps:

installed 1.63.124
launched Brave using --env-leo=staging --env-ai-chat.bsg=dev --env-ai-chat-premium.bsg=dev --enable-logging=stderr
clicked on Leo in the sidebar
clicked on Accept and begin
loaded wired.com
loaded an article
clicked on Summarize this page
clicked on Suggest questions...
opened brave://settings/leo-assistant

Confirmed `Mixtral` as the non-Premium default

example	example	example	example

1st launch	2nd launch	default

New Premium-enabled indicator: brave/brave-browser#34718 - `PASSED`

Steps:

installed 1.63.124
launched Brave using --env-leo=staging --env-ai-chat.bsg=dev --env-ai-chat-premium.bsg=dev --enable-logging=stderr
clicked on Leo in the sidebar
clicked on Accept and begin
clicked on the ... menu at the top right

Confirmed the new `Limited` icon

`Dark` mode	`Light` mode

) * AI Chat: introduce freemium model concept (#21398) * AI Chat: introduce freemium model concept * AI Chat: Add mixtral model * AI Chat: premium's default also becomes leo-expanded * AI Chat: non-premium users on freemium models get rate limit error which prompts to use a basic (free) model as secondary action * fix format from chromium upgrade * AI Chat: introduce "is_freemium_available" boolean feature param If true, certain freemium models are available to non-premium users. If false, those models are premium-only. * AI Chat: migrate default model key pref from "chat-default" to "chat-basic" * AI Chat: reduce claude page content character length to avoid token length mismatch more often * AI Chat: model menu uses model display name instead of API name * fix GetModels * format

@petemill

Related to brave/brave-core#21398 Blocked on deploying brave/aichat-ops#359 to prod This sets the default model for 25% of free users to `chat-basic`, which corresponds to [Llama 3 8b](https://github.com/brave/brave-core/blob/c839e1676031a3d155b04b9dbfe49e11b3b8601b/components/ai_chat/core/browser/model_service.cc#L154). The intention is to start this to progressively roll out this change (25%, 50%, 75%, 100%), so we can ensure our single instance of Llama 3 can handle the increase in traffic. Based on these [estimations](https://artificialanalysis.ai/?models_selected=llama-3-1-instruct-8b%2Cmixtral-8x7b-instruct), one instance of llama 3 (one gpu) should have higher token throughput than our single instance of mixtral (four gpus), so we expect this to work with two llama instances which will be added in brave/aichat-ops#359. cc @petemill @LorenzoMinto Note: * Chromium version 122.0.6261.57 was selected because that was the first chromium version when 1.63.x was released (see https://bravesoftware.slack.com/archives/C04PX1BUN/p1708629893634639), which is when brave/brave-core#21398 went in. * I have included all platforms because we want to make this change across all platforms, however this does differ from the BraveAIChatEnabledStudy which applies only to desktop --------- Co-authored-by: Nick von Pentz <nvonpentz@users.noreply.github.com> Co-authored-by: Pete Miller <miller.pete@gmail.com> Co-authored-by: Aleksey Khoroshilov <akhoroshilov@brave.com>

petemill self-assigned this Dec 15, 2023

github-actions bot added the CI/storybook-url Deploy storybook and provide a unique URL for each build label Dec 15, 2023

petemill force-pushed the ai-chat-freemium branch from f565bc7 to 28d69f8 Compare December 15, 2023 10:11

petemill force-pushed the ai-chat-freemium branch from 28d69f8 to eaf4886 Compare December 29, 2023 21:33

niutech mentioned this pull request Dec 31, 2023

Support Mixtral-8x7b when it's available aandrew-me/tgpt#182

Closed

petemill marked this pull request as ready for review January 8, 2024 20:41

petemill requested a review from a team as a code owner January 8, 2024 20:41

petemill requested a review from nullhook January 8, 2024 20:41

petemill force-pushed the ai-chat-freemium branch from eaf4886 to 7838ad5 Compare January 9, 2024 03:02

petemill added 9 commits January 15, 2024 19:44

AI Chat: introduce freemium model concept

6e33d05

AI Chat: Add mixtral model

371307a

Basic -> Limited

181be90

format

120d178

AI Chat: premium's default also becomes leo-expanded

4aac43b

AI Chat: non-premium users on freemium models get rate limit error wh…

17e6c25

…ich prompts to use a basic (free) model as secondary action

fix format from chromium upgrade

9623be4

AI Chat: introduce "is_freemium_available" boolean feature param

2d31274

If true, certain freemium models are available to non-premium users. If false, those models are premium-only.

AI Chat: migrate default model key pref from "chat-default" to "chat-…

b7be646

…basic"

petemill force-pushed the ai-chat-freemium branch from f4fa11c to b7be646 Compare January 16, 2024 08:39

github-actions bot added the puLL-Merge label Jan 16, 2024

petemill commented Jan 16, 2024

View reviewed changes

petemill added 2 commits January 16, 2024 01:08

optimize

9d8f988

label style and translate

60f62d1

petemill force-pushed the ai-chat-freemium branch from 403ca6f to 60f62d1 Compare January 16, 2024 09:08

Back to GetCurrentModel returning a reference as it can't be null and…

0b3a41d

… maintains iOS compat

nullhook reviewed Jan 16, 2024

View reviewed changes

nullhook self-requested a review January 16, 2024 17:07

nullhook approved these changes Jan 16, 2024

View reviewed changes

petemill added 2 commits January 16, 2024 10:48

AI Chat: reduce claude page content character length to avoid token l…

3e146b0

…ength mismatch more often

AI Chat: model menu uses model display name instead of API name

12beb00

bsclifton approved these changes Jan 16, 2024

View reviewed changes

petemill merged commit c816be6 into master Jan 16, 2024
18 checks passed

petemill deleted the ai-chat-freemium branch January 16, 2024 22:08

github-actions bot added this to the 1.63.x - Nightly milestone Jan 16, 2024

petemill mentioned this pull request Jan 16, 2024

[Uplift 1.62.x] #21398 AI Chat: introduce freemium model concept #21594

Merged

kjozwiak mentioned this pull request Jan 19, 2024

Add griffin feature flag for setting Llama13B as the default brave/brave-browser#34721

Closed

petemill mentioned this pull request Apr 8, 2024

aichat: enable migration code #22965

Merged

24 tasks

nvonpentz mentioned this pull request Oct 15, 2024

Set Leo default model to Llama 3 8b (25% rollout) brave/brave-variations#1225

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AI Chat: introduce freemium model concept #21398

AI Chat: introduce freemium model concept #21398

petemill commented Dec 15, 2023 •

edited

Loading

brave-builds commented Dec 15, 2023

brave-builds commented Dec 15, 2023

brave-builds commented Dec 29, 2023

brave-builds commented Jan 9, 2024

brave-builds commented Jan 9, 2024

petemill Jan 16, 2024 •

edited

Loading

petemill Jan 16, 2024

brave-builds commented Jan 16, 2024

nullhook Jan 16, 2024 •

edited

Loading

nullhook Jan 16, 2024 •

edited

Loading

github-actions bot commented Jan 16, 2024

Changes

bsclifton left a comment

brave-builds commented Jan 16, 2024

stephendonner commented Jan 17, 2024 •

edited

Loading

`Mixtral` - `PASSED`

Steps:

`Claude Instant` - `PASSED`

Steps:

`llama2-13b` - `PASSED`

Steps:

Confirmed `Mixtral` as the non-Premium default

Steps:

Confirmed the new `Limited` icon

AI Chat: introduce freemium model concept #21398

AI Chat: introduce freemium model concept #21398

Conversation

petemill commented Dec 15, 2023 • edited Loading

Don't allow freemium models to be selected when griffin flag isn't pointing at a freemium model (or other flag)

Submitter Checklist:

Reviewer Checklist:

After-merge Checklist:

Test Plan:

brave-builds commented Dec 15, 2023

brave-builds commented Dec 15, 2023

brave-builds commented Dec 29, 2023

brave-builds commented Jan 9, 2024

brave-builds commented Jan 9, 2024

petemill Jan 16, 2024 • edited Loading

Choose a reason for hiding this comment

petemill Jan 16, 2024

Choose a reason for hiding this comment

brave-builds commented Jan 16, 2024

nullhook Jan 16, 2024 • edited Loading

Choose a reason for hiding this comment

nullhook Jan 16, 2024 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Jan 16, 2024

Description

Changes

Security Hotspots

bsclifton left a comment

Choose a reason for hiding this comment

brave-builds commented Jan 16, 2024

stephendonner commented Jan 17, 2024 • edited Loading

Verification PASSED using

Allow some Premium models to be used by non-Premium users brave/brave-browser#34910 - PASSED

Mixtral - PASSED

Steps:

Claude Instant - PASSED

Steps:

llama2-13b - PASSED

Add Griffin feature flag for setting Llama-13b (actually Mixtral) as the default: brave/brave-browser#34721 - PASSED

Steps:

Confirmed Mixtral as the non-Premium default

New Premium-enabled indicator: brave/brave-browser#34718 - PASSED

Steps:

Confirmed the new Limited icon

petemill commented Dec 15, 2023 •

edited

Loading

petemill Jan 16, 2024 •

edited

Loading

nullhook Jan 16, 2024 •

edited

Loading

nullhook Jan 16, 2024 •

edited

Loading

stephendonner commented Jan 17, 2024 •

edited

Loading

Verification `PASSED` using

Allow some Premium models to be used by non-Premium users brave/brave-browser#34910 - `PASSED`

`Mixtral` - `PASSED`

`Claude Instant` - `PASSED`

`llama2-13b` - `PASSED`

Add Griffin feature flag for setting `Llama-13b` (actually `Mixtral`) as the default: brave/brave-browser#34721 - `PASSED`

Confirmed `Mixtral` as the non-Premium default

New Premium-enabled indicator: brave/brave-browser#34718 - `PASSED`

Confirmed the new `Limited` icon