Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encode model in GenerateContentRequest only when needed #174

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Encode model in GenerateContentRequest only when needed #174

wants to merge 2 commits into from

Conversation

andrewheard
Copy link
Collaborator

@andrewheard andrewheard commented May 22, 2024 •
edited
Loading

Updated the GenerateContentRequest encoding implementation to serialize the model field only when needed (currently only needed for countTokens).

@github-actions github-actions bot added component:swift sdk Issue/PR related to Swift SDK status:awaiting review PR awaiting review from a maintainer labels May 22, 2024
@andrewheard andrewheard requested a review from ncooke3 May 22, 2024 20:04
@ncooke3
Copy link
Collaborator

ncooke3 commented May 22, 2024

Updated the countTokens implementation to populate generateContentRequest instead of contents in the request body.

  • This provides a more accurate token count since, e.g., function declarations and system instructions from the request are now counted.

Does it imply that the SDK currently only counts tokens from the prompt?

@andrewheard
Copy link
Collaborator Author

Updated the countTokens implementation to populate generateContentRequest instead of contents in the request body.

  • This provides a more accurate token count since, e.g., function declarations and system instructions from the request are now counted.

Does it imply that the SDK currently only counts tokens from the prompt?

@ncooke3 That's accurate. Prior to this PR the SDK would only count tokens from the prompt itself.

ncooke3
ncooke3 previously approved these changes May 22, 2024
View reviewed changes
Sources/GoogleAI/CountTokensRequest.swift Show resolved Hide resolved
Sources/GoogleAI/GenerativeModel.swift Show resolved Hide resolved
@andrewheard andrewheard mentioned this pull request May 22, 2024
Merged
@andrewheard andrewheard marked this pull request as draft May 28, 2024 22:04
@andrewheard andrewheard force-pushed the ah/count-tokens-generate branch from 324e2f8 to 8a0c9bb Compare May 28, 2024 22:21
@andrewheard andrewheard changed the title Send GenerateContentRequest in CountTokensRequest Encode model in GenerateContentRequest only when needed May 28, 2024
@andrewheard andrewheard changed the base branch from main to ah/count-tokens-generate-alt May 28, 2024 22:25
@andrewheard andrewheard changed the base branch from ah/count-tokens-generate-alt to main May 28, 2024 22:27
@andrewheard andrewheard dismissed ncooke3’s stale review May 28, 2024 22:27

The base branch was changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ncooke3 ncooke3 ncooke3 left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
component:swift sdk Issue/PR related to Swift SDK status:awaiting review PR awaiting review from a maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andrewheard @ncooke3