feat(vertexai)!: Vertex AI in Firebase is now Generally Available (GA…

…) and can be used in production apps. (#13453) * chore(vertexai): fork vertexai sdk away from generative ai sdk. (#13298) * fix the sdk name and support platform * forking off model and chat class * fork api class * remove some export content since the next vertexai release will be breaking change. * fork function calling * fork off content * fix for analyzer * more analyzer fix * chore(vertexai): Add corresponding unit test for the fork (#13324) * Add corresponding unit test for the fork * fix issues from analyzer * feat(vertexai): add json schema support and update schema structure (#13378) * Add schema support for prompt response * update schema object, replace requiredProperties to optionalProperties * api updates (breaking) * more tweak during api review * Add format for Schema.string * fix test files * Update packages/firebase_vertexai/firebase_vertexai/lib/src/vertex_api.dart Co-authored-by: Nate Bosch <nbosch@google.com> * add review feedback --------- Co-authored-by: Nate Bosch <nbosch@google.com> * feat(vertexai): update function calling and sample (#13395) * Add schema support for prompt response * update schema object, replace requiredProperties to optionalProperties * api updates (breaking) * more tweak during api review * Add format for Schema.string * fix test files * first sample update * remove embedding * api update for function calling * Update sample code to match the new api * more code snippet for toolconfig * apply documentation update * tweak of the tools * remove embed related class * Update packages/firebase_vertexai/firebase_vertexai/example/lib/main.dart Co-authored-by: Nate Bosch <nbosch@google.com> * Update packages/firebase_vertexai/firebase_vertexai/example/lib/main.dart Co-authored-by: Nate Bosch <nbosch@google.com> * address review comments * more fix * keep name consistency * keep using getWeather to keep align with devsite documentation * Update the static name --------- Co-authored-by: Nate Bosch <nbosch@google.com> * feat(vertexai): update enums and citation and DataPart (#13410) * Update SafeRating * Rename citation source * add blocked for SafetyRating parse * Rename DataPart to InlineDataPart * Update readme * apply review comment * feat(vertexai): Error message for service api not enabled. (#13435) * Error message for service api not enabled. * add quota exceed test * Apply suggestions from code review Co-authored-by: Nate Bosch <nbosch@google.com> * Update for review comments --------- Co-authored-by: Nate Bosch <nbosch@google.com> * chore(vertexai): remove vertex_ file name for most of files (#13439) * remove vertex_ file name for most of files * organize import * fix the optional attributes while parsing SafetyRating (#13452) * chore(vertexai): function calling sample update (#13492) * fix the optional attributes while parsing SafetyRating * add doc comments * update function sample * comments * Shorten lines over 80 columns (#13493) Reflow comments and break long strings into adjacent string literals. --------- Co-authored-by: Nate Bosch <nbosch@google.com>
firebase · Oct 18, 2024 · 77b4880 · 77b4880
1 parent f010b46
commit 77b4880
Show file tree

Hide file tree

Showing 29 changed files with 3,321 additions and 1,362 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1926,7 +1926,7 @@ Packages with other changes:
 
 #### `firebase_vertexai` - `v0.1.0`
 
- - Initial release of the Vertex AI for Firebase SDK (public preview). Learn how to [get started](https://firebase.google.com/docs/vertex-ai/get-started) with the SDK in your app.
+ - Initial release of the Vertex AI in Firebase SDK (public preview). Learn how to [get started](https://firebase.google.com/docs/vertex-ai/get-started) with the SDK in your app.
 
 
 ## 2024-05-07

diff --git a/packages/firebase_app_check/firebase_app_check_web/pubspec.yaml b/packages/firebase_app_check/firebase_app_check_web/pubspec.yaml
@@ -1,5 +1,5 @@
 name: firebase_app_check_web
-description: The web implementation of firebase_auth
+description: The web implementation of firebase_app_check
 homepage: https://github.com/firebase/flutterfire/tree/main/packages/firebase_app_check/firebase_app_check_web
 version: 0.1.3+2
 

diff --git a/packages/firebase_vertexai/firebase_vertexai/CHANGELOG.md b/packages/firebase_vertexai/firebase_vertexai/CHANGELOG.md
@@ -62,4 +62,4 @@
 
 ## 0.1.0
 
-- Initial release of the Vertex AI for Firebase SDK (public preview). Learn how to [get started](https://firebase.google.com/docs/vertex-ai/get-started) with the SDK in your app.
+- Initial release of the Vertex AI in Firebase SDK (public preview). Learn how to [get started](https://firebase.google.com/docs/vertex-ai/get-started) with the SDK in your app.
diff --git a/packages/firebase_vertexai/firebase_vertexai/README.md b/packages/firebase_vertexai/firebase_vertexai/README.md
@@ -1,15 +1,15 @@
-# Vertex AI for Firebase Flutter
+# Vertex AI in Firebase Flutter
 [![pub package](https://img.shields.io/pub/v/firebase_vertexai.svg)](https://pub.dev/packages/firebase_vertexai)
 
 A Flutter plugin to use the [Vertex AI](https://firebase.google.com/docs/vertex-ai/).
 
 To learn more about Vertex AI, please visit the [website](https://cloud.google.com/vertex-ai)
 
-**Preview**: Vertex AI for Firebase is in Public Preview, which means that the product is not subject to any SLA or deprecation policy and could change in backwards-incompatible ways.
+**Generally Available**: Vertex AI in Firebase is now Generally Available (GA) and can be used in production apps
 
 ## Getting Started
 
-To get started with Vertex AI for Firebase Flutter, please [see the documentation](https://firebase.google.com/docs/vertex-ai/get-started?platform=flutter).
+To get started with Vertex AI in Firebase Flutter, please [see the documentation](https://firebase.google.com/docs/vertex-ai/get-started?platform=flutter).
 
 ## Usage
 

diff --git a/packages/firebase_vertexai/firebase_vertexai/example/README.md b/packages/firebase_vertexai/firebase_vertexai/example/README.md
@@ -1,6 +1,6 @@
 # firebase_vertexai_example
 
-Sample app to show how to use Vertex AI for Firebase.
+Sample app to show how to use Vertex AI in Firebase.
 
 ## Getting Started
 

diff --git a/packages/firebase_vertexai/firebase_vertexai/example/lib/main.dart b/packages/firebase_vertexai/firebase_vertexai/example/lib/main.dart
@@ -91,52 +91,66 @@ class _ChatWidgetState extends State<ChatWidget> {
 
     initFirebase().then((value) {
       _model = FirebaseVertexAI.instance.generativeModel(
-        model: 'gemini-1.5-flash-preview-0514',
+        model: 'gemini-1.5-flash',
       );
       _functionCallModel = FirebaseVertexAI.instance.generativeModel(
-        model: 'gemini-1.5-flash-preview-0514',
+        model: 'gemini-1.5-flash',
         tools: [
-          Tool(functionDeclarations: [exchangeRateTool]),
+          Tool.functionDeclarations([fetchWeatherTool]),
         ],
       );
       _chat = _model.startChat();
     });
   }
 
-  Future<Map<String, Object?>> findExchangeRate(
-    Map<String, Object?> arguments,
-  ) async =>
-      // This hypothetical API returns a JSON such as:
-      // {"base":"USD","date":"2024-04-17","rates":{"SEK": 0.091}}
-      {
-        'date': arguments['currencyDate'],
-        'base': arguments['currencyFrom'],
-        'rates': <String, Object?>{arguments['currencyTo']! as String: 0.091},
-      };
-
-  final exchangeRateTool = FunctionDeclaration(
-    'findExchangeRate',
-    'Returns the exchange rate between currencies on given date.',
-    Schema(
-      SchemaType.object,
-      properties: {
-        'currencyDate': Schema(
-          SchemaType.string,
-          description: 'A date in YYYY-MM-DD format or '
-              'the exact value "latest" if a time period is not specified.',
-        ),
-        'currencyFrom': Schema(
-          SchemaType.string,
-          description: 'The currency code of the currency to convert from, '
-              'such as "USD".',
-        ),
-        'currencyTo': Schema(
-          SchemaType.string,
-          description: 'The currency code of the currency to convert to, '
-              'such as "USD".',
-        ),
-      },
-    ),
+  // This is a hypothetical API to return a fake weather data collection for
+  // certain location
+  Future<Map<String, Object?>> fetchWeather(
+    double latitude,
+    double longitude,
+    String date,
+  ) async {
+    // TODO(developer): Call a real weather API.
+    // Mock response from the API. In developer live code this would call the
+    // external API and return what that API returns.
+    final apiResponse = {
+      'location': '$latitude, $longitude',
+      'date': date,
+      'temperature': 38,
+      'chancePrecipitation': '56%',
+      'cloudConditions': 'partly-cloudy',
+    };
+    return apiResponse;
+  }
+
+  /// Actual function to demonstrate the function calling feature.
+  final fetchWeatherTool = FunctionDeclaration(
+    'fetchWeather',
+    'Get the weather conditions for a specific city on a specific date.',
+    parameters: {
+      'location': Schema.object(
+        description: 'The longitude and latitude of the city for which to get '
+            'the weather. Must always be a nested object of '
+            '`longitude` and `latitude`. The values must be floats.',
+        properties: {
+          'latitude': Schema.number(
+            format: 'float',
+            description: 'A numeric value indicating the latitude of the '
+                'desired location between -90 and 90',
+          ),
+          'longitude': Schema.number(
+            format: 'float',
+            description:
+                'A numeric value indicating the longitude of the desired '
+                'location between -180 and 180',
+          ),
+        },
+      ),
+      'date': Schema.string(
+        description: 'The date for which to get the weather. '
+            'Date must be in the format: YYYY-MM-DD.',
+      ),
+    },
   );
 
   Future<void> initFirebase() async {
@@ -274,6 +288,20 @@ class _ChatWidgetState extends State<ChatWidget> {
                         : Theme.of(context).colorScheme.primary,
                   ),
                 ),
+                IconButton(
+                  tooltip: 'schema prompt',
+                  onPressed: !_loading
+                      ? () async {
+                          await _promptSchemaTest(_textController.text);
+                        }
+                      : null,
+                  icon: Icon(
+                    Icons.schema,
+                    color: _loading
+                        ? Theme.of(context).colorScheme.secondary
+                        : Theme.of(context).colorScheme.primary,
+                  ),
+                ),
                 if (!_loading)
                   IconButton(
                     onPressed: () async {
@@ -304,6 +332,51 @@ class _ChatWidgetState extends State<ChatWidget> {
     );
   }
 
+  Future<void> _promptSchemaTest(String subject) async {
+    setState(() {
+      _loading = true;
+    });
+    try {
+      final content = [Content.text('Create a list of 20 $subject.')];
+
+      final response = await _model.generateContent(
+        content,
+        generationConfig: GenerationConfig(
+          responseMimeType: 'application/json',
+          responseSchema: Schema.array(
+            items: Schema.string(
+              description: 'A single word that a player will need to guess.',
+            ),
+          ),
+        ),
+      );
+
+      var text = response.text;
+      _generatedContent.add((image: null, text: text, fromUser: false));
+
+      if (text == null) {
+        _showError('No response from API.');
+        return;
+      } else {
+        setState(() {
+          _loading = false;
+          _scrollDown();
+        });
+      }
+    } catch (e) {
+      _showError(e.toString());
+      setState(() {
+        _loading = false;
+      });
+    } finally {
+      _textController.clear();
+      setState(() {
+        _loading = false;
+      });
+      _textFieldFocus.requestFocus();
+    }
+  }
+
   Future<void> _sendStorageUriPrompt(String message) async {
     setState(() {
       _loading = true;
@@ -358,8 +431,8 @@ class _ChatWidgetState extends State<ChatWidget> {
         Content.multi([
           TextPart(message),
           // The only accepted mime types are image/*.
-          DataPart('image/jpeg', catBytes.buffer.asUint8List()),
-          DataPart('image/jpeg', sconeBytes.buffer.asUint8List()),
+          InlineDataPart('image/jpeg', catBytes.buffer.asUint8List()),
+          InlineDataPart('image/jpeg', sconeBytes.buffer.asUint8List()),
         ]),
       ];
       _generatedContent.add(
@@ -444,29 +517,35 @@ class _ChatWidgetState extends State<ChatWidget> {
     setState(() {
       _loading = true;
     });
-    final chat = _functionCallModel.startChat();
-    const prompt = 'How much is 50 US dollars worth in Swedish krona?';
+    final functionCallChat = _functionCallModel.startChat();
+    const prompt = 'What is the weather like in Boston on 10/02 this year?';
 
     // Send the message to the generative model.
-    var response = await chat.sendMessage(Content.text(prompt));
+    var response = await functionCallChat.sendMessage(
+      Content.text(prompt),
+    );
 
     final functionCalls = response.functionCalls.toList();
     // When the model response with a function call, invoke the function.
     if (functionCalls.isNotEmpty) {
       final functionCall = functionCalls.first;
-      final result = switch (functionCall.name) {
-        // Forward arguments to the hypothetical API.
-        'findExchangeRate' => await findExchangeRate(functionCall.args),
-        // Throw an exception if the model attempted to call a function that was
-        // not declared.
-        _ => throw UnimplementedError(
-            'Function not implemented: ${functionCall.name}',
-          )
-      };
-      // Send the response to the model so that it can use the result to generate
-      // text for the user.
-      response = await chat
-          .sendMessage(Content.functionResponse(functionCall.name, result));
+      if (functionCall.name == 'fetchWeather') {
+        Map<String, dynamic> location =
+            functionCall.args['location']! as Map<String, dynamic>;
+        var date = functionCall.args['date']! as String;
+        var latitude = location['latitude']! as double;
+        var longitude = location['longitude']! as double;
+        final functionResult = await fetchWeather(latitude, longitude, date);
+        // Send the response to the model so that it can use the result to
+        // generate text for the user.
+        response = await functionCallChat.sendMessage(
+          Content.functionResponse(functionCall.name, functionResult),
+        );
+      } else {
+        throw UnimplementedError(
+          'Function not declared to the model: ${functionCall.name}',
+        );
+      }
     }
     // When the model responds with non-null text content, print it.
     if (response.text case final text?) {
@@ -483,11 +562,21 @@ class _ChatWidgetState extends State<ChatWidget> {
     });
 
     const prompt = 'tell a short story';
-    var response = await _model.countTokens([Content.text(prompt)]);
-    print(
-      'token: ${response.totalTokens}, billable characters: ${response.totalBillableCharacters}',
-    );
-
+    var content = Content.text(prompt);
+    var tokenResponse = await _model.countTokens([content]);
+    final tokenResult = 'Count token: ${tokenResponse.totalTokens}, billable '
+        'characters: ${tokenResponse.totalBillableCharacters}';
+    _generatedContent.add((image: null, text: tokenResult, fromUser: false));
+
+    var contentResponse = await _model.generateContent([content]);
+    final contentMetaData = 'result metadata, promptTokenCount:'
+        '${contentResponse.usageMetadata!.promptTokenCount}, '
+        'candidatesTokenCount:'
+        '${contentResponse.usageMetadata!.candidatesTokenCount}, '
+        'totalTokenCount:'
+        '${contentResponse.usageMetadata!.totalTokenCount}';
+    _generatedContent
+        .add((image: null, text: contentMetaData, fromUser: false));
     setState(() {
       _loading = false;
     });

diff --git a/packages/firebase_vertexai/firebase_vertexai/lib/firebase_vertexai.dart b/packages/firebase_vertexai/firebase_vertexai/lib/firebase_vertexai.dart
@@ -12,25 +12,13 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-export 'src/firebase_vertexai.dart'
+export 'src/api.dart'
     show
-        // TODO(next breaking): Remove defaultTimeout
-        defaultTimeout,
-        FirebaseVertexAI,
-        RequestOptions;
-export 'src/vertex_api.dart'
-    show
-        BatchEmbedContentsResponse,
         BlockReason,
         Candidate,
         CitationMetadata,
-        CitationSource,
-        ContentEmbedding,
+        Citation,
         CountTokensResponse,
-        // TODO(next breaking): Remove CountTokensResponseFields
-        CountTokensResponseFields,
-        EmbedContentRequest,
-        EmbedContentResponse,
         FinishReason,
         GenerateContentResponse,
         GenerationConfig,
@@ -41,29 +29,31 @@ export 'src/vertex_api.dart'
         SafetyRating,
         SafetySetting,
         TaskType,
-        // TODO(next breaking): Remove parse* methods
-        parseCountTokensResponse,
-        parseEmbedContentResponse,
-        parseGenerateContentResponse;
-export 'src/vertex_chat.dart' show ChatSession, StartChatExtension;
-export 'src/vertex_content.dart'
+        UsageMetadata;
+export 'src/chat.dart' show ChatSession, StartChatExtension;
+export 'src/content.dart'
     show
         Content,
-        DataPart,
+        InlineDataPart,
         FileData,
         FunctionCall,
         FunctionResponse,
         Part,
-        TextPart,
-        // TODO(next breaking): Remove parseContent
-        parseContent;
-export 'src/vertex_function_calling.dart'
+        TextPart;
+export 'src/error.dart'
+    show
+        VertexAIException,
+        VertexAISdkException,
+        InvalidApiKey,
+        ServerException,
+        UnsupportedUserLocation;
+export 'src/firebase_vertexai.dart' show FirebaseVertexAI;
+export 'src/function_calling.dart'
     show
         FunctionCallingConfig,
         FunctionCallingMode,
         FunctionDeclaration,
-        Schema,
-        SchemaType,
         Tool,
         ToolConfig;
-export 'src/vertex_model.dart' show GenerativeModel;
+export 'src/model.dart' show GenerativeModel;
+export 'src/schema.dart' show Schema, SchemaType;
Original file line number	Diff line number	Diff line change
Expand Up		@@ -62,4 +62,4 @@

		## 0.1.0

		- Initial release of the Vertex AI for Firebase SDK (public preview). Learn how to [get started](https://firebase.google.com/docs/vertex-ai/get-started) with the SDK in your app.
		- Initial release of the Vertex AI in Firebase SDK (public preview). Learn how to [get started](https://firebase.google.com/docs/vertex-ai/get-started) with the SDK in your app.