Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Voyage AI support #575

Open
wants to merge 28 commits into
base: main
Choose a base branch
from

Conversation

devin-ai-integration[bot]
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Dec 12, 2024

Voyage AI Integration Updates

Changes

  • Fixed event data handling in VoyageProvider
  • Added proper population of Prompt, Completion, Params, and Returns fields
  • Improved debug logging for event verification
  • Applied code formatting to all Voyage AI integration files

Testing

Tested locally with mock client
Verified event data fields are properly populated:

  • Prompt: Contains input text
  • Completion: Contains embedding vector (1024-dimensional)
  • Params: Includes original kwargs
  • Returns: Contains complete API response

Session Verification

Latest test session: https://app.agentops.ai/drilldown?session_id=174f739f-30bf-4a1d-933a-2a30b10e7000

Link to Devin run: https://app.devin.ai/sessions/97458294b13e45849dea480c9ed52ef7

Closes #461

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
@devin-ai-integration devin-ai-integration bot added the enhancement New feature or request label Dec 12, 2024
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR
  • Look at CI failures and help fix them

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Add "(aside)" to your comment to have me ignore it.

Copy link

codecov bot commented Dec 12, 2024

Codecov Report

Attention: Patch coverage is 52.87356% with 41 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
agentops/llms/providers/voyage.py 34.92% 40 Missing and 1 partial ⚠️
Flag Coverage Δ
unittests 54.82% <52.87%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
tests/test_session.py 98.68% <100.00%> (+0.08%) ⬆️
agentops/llms/providers/voyage.py 34.92% <34.92%> (ø)

... and 1 file with indirect coverage changes

devin-ai-integration bot and others added 12 commits December 12, 2024 06:31
…tests

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
…r handling

- Add async embedding support
- Improve error handling and logging
- Add token counting from usage data
- Enhance type hints
- Maintain Python version warning

Fixes #461

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
- Replace incorrect [external] and [tool.tach.dependencies] with [dependency-group.ci]
- Keep existing module configurations intact
- Fix 'Group ci is not defined' error

Part of #461

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Copy link

gitguardian bot commented Dec 14, 2024

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
14864488 Triggered Generic High Entropy Secret ad922f8 examples/voyage/voyage_example.py View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Comment on lines 29 to 38
def main():
# Set AgentOps API key
os.environ["AGENTOPS_API_KEY"] = "8b95388c-ee56-499d-a940-c1d6a2ba7f0c"

# Initialize clients
voyage_client = MockVoyageClient()
ao_client = AgentopsClient()

# Initialize session
session = ao_client.initialize()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Bug Fix:

Avoid Hardcoding API Keys
Hardcoding API keys directly in the code poses a significant security risk. It can lead to unauthorized access if the code is exposed in version control or logs.

  • Store the API key in a secure environment variable or a configuration file that is not included in version control.
  • Access the API key using os.environ or a secure configuration management tool.
  • This approach enhances security and prevents potential misuse of the API. 🔒

🔧 Suggested Code Diff:

import os

def main():
    # Set AgentOps API key from environment variable
    api_key = os.getenv("AGENTOPS_API_KEY")
    if not api_key:
        raise ValueError("API key not found. Please set the AGENTOPS_API_KEY environment variable.")
    os.environ["AGENTOPS_API_KEY"] = api_key

    # Initialize clients
    voyage_client = MockVoyageClient()
    ao_client = AgentopsClient()

    # Initialize session
    session = ao_client.initialize()
    print(f"Session URL: {session.session_url}")
📝 Committable Code Suggestion

‼️ Ensure you review the code suggestion before committing it to the branch. Make sure it replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def main():
# Set AgentOps API key
os.environ["AGENTOPS_API_KEY"] = "8b95388c-ee56-499d-a940-c1d6a2ba7f0c"
# Initialize clients
voyage_client = MockVoyageClient()
ao_client = AgentopsClient()
# Initialize session
session = ao_client.initialize()
import os
def main():
# Set AgentOps API key from environment variable
api_key = os.getenv("AGENTOPS_API_KEY")
if not api_key:
raise ValueError("API key not found. Please set the AGENTOPS_API_KEY environment variable.")
os.environ["AGENTOPS_API_KEY"] = api_key
# Initialize clients
voyage_client = MockVoyageClient()
ao_client = AgentopsClient()
# Initialize session
session = ao_client.initialize()
print(f"Session URL: {session.session_url}")
📜 Guidelines
  • Use exceptions for error handling, but avoid assert statements for critical logic.

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Comment on lines 29 to 40
def main():
# Set AgentOps API key
os.environ["AGENTOPS_API_KEY"] = "8b95388c-ee56-499d-a940-c1d6a2ba7f0c"

# Initialize clients
voyage_client = MockVoyageClient()
ao_client = AgentopsClient()

# Initialize session
session = ao_client.initialize()
print(f"Session URL: {session.session_url}")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Bug Fix:

Remove Hardcoded API Key
Hardcoding API keys in the code is a significant security risk. It can lead to unauthorized access if the code is exposed. Instead, retrieve the API key from a secure environment variable. This approach enhances security and makes the code more flexible for different environments.

🔧 Suggested Code Diff:

-def main():
-    # Set AgentOps API key
-    os.environ["AGENTOPS_API_KEY"] = "8b95388c-ee56-499d-a940-c1d6a2ba7f0c"
+def main():
+    # Retrieve AgentOps API key from environment variable
+    api_key = os.environ.get("AGENTOPS_API_KEY")
+    if not api_key:
+        raise ValueError("AGENTOPS_API_KEY environment variable not set")
+    os.environ["AGENTOPS_API_KEY"] = api_key

    # Initialize clients
    voyage_client = MockVoyageClient()
    ao_client = AgentopsClient()
📝 Committable Code Suggestion

‼️ Ensure you review the code suggestion before committing it to the branch. Make sure it replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def main():
# Set AgentOps API key
os.environ["AGENTOPS_API_KEY"] = "8b95388c-ee56-499d-a940-c1d6a2ba7f0c"
# Initialize clients
voyage_client = MockVoyageClient()
ao_client = AgentopsClient()
# Initialize session
session = ao_client.initialize()
print(f"Session URL: {session.session_url}")
def main():
# Retrieve AgentOps API key from environment variable
api_key = os.environ.get("AGENTOPS_API_KEY")
if not api_key:
raise ValueError("AGENTOPS_API_KEY environment variable not set")
os.environ["AGENTOPS_API_KEY"] = api_key
# Initialize clients
voyage_client = MockVoyageClient()
ao_client = AgentopsClient()
# Initialize session
session = ao_client.initialize()
print(f"Session URL: {session.session_url}")
📜 Guidelines

Python: Avoid mutable global states


devin-ai-integration bot and others added 8 commits December 14, 2024 04:24
- Apply automatic formatting fixes from ruff
- Ensure code meets project style guidelines

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Comment on lines 39 to 47
def record(self, event):
"""Record event with minimal data exposure."""
try:
event_data = {
"type": "llms",
"model": "default",
"input_tokens": 10,
"output_tokens": 0,
"input": getattr(event, "input", ""),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Bug Fix:

Syntax Error Due to Missing Comma
There is a missing comma between the 'input_tokens' and 'output_tokens' keys in the dictionary definition within the record method. This will cause a syntax error, preventing the code from executing correctly. Please add the missing comma to resolve this issue. 🛠️

🔧 Suggested Code Diff:

            event_data = {
                "type": "llms",
                "model": "default",
                "input_tokens": 10,
                "output_tokens": 0,
                "input": getattr(event, "input", ""),
                "output": {
                    "type": "embedding",
                    "data": ["<vector data redacted>"],
                },
                "metadata": {"text": getattr(event, "input", "")},
            }
📝 Committable Code Suggestion

‼️ Ensure you review the code suggestion before committing it to the branch. Make sure it replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def record(self, event):
"""Record event with minimal data exposure."""
try:
event_data = {
"type": "llms",
"model": "default",
"input_tokens": 10,
"output_tokens": 0,
"input": getattr(event, "input", ""),
def record(self, event):
"""Record event with minimal data exposure."""
try:
event_data = {
"type": "llms",
"model": "default",
"input_tokens": 10,
"output_tokens": 0,
"input": getattr(event, "input", ""),
"output": {
"type": "embedding",
"data": ["<vector data redacted>"],
},
"metadata": {"text": getattr(event, "input", "")},
}
# Further processing of event_data can be done here
except Exception as e:
# Handle exceptions appropriately
print(f"An error occurred: {e}")
🔍 Learnings

NA

📜 Guidelines

• Follow PEP 8 style guidelines


Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Voyage AI support
1 participant