🐛 fix: Anthropic Prompt Caching Edge Case

[danny-avila]

Summary

I addressed an edge case in the Anthropic prompt caching mechanism, specifically improving the handling of multi-modal messages and ensuring proper cache control for text content.

• Refactored the addCacheControl function to handle complex message structures more effectively.
• Modified the logic to only add cache control to text content within user messages.
• Updated the function to process up to two user messages, focusing on the most recent ones.
• Improved efficiency by breaking the inner loop once a text content is processed in array-type messages.
• Enhanced test coverage to include edge cases with multiple content types and messages without text blocks.
• Ensured that non-text content (like images) are not modified with cache control properties.

Anthropic only allows 4 items to have the cache_control property per payload. Our method now correctly stays under this limit with a maximum of 3 (last 2 user messages, and +1 if system prompt is included)

Testing

To test these changes:

1. Run the updated unit tests in addCacheControl.spec.js.
2. Manually test the chat functionality with Anthropic models, particularly with multi-modal messages (text + images).
3. Verify that only text content in user messages receives the cache_control property.
4. Check that the changes do not affect the behavior of other message types or roles.

Test Configuration:

• Node.js environment
• Jest for running unit tests
• Anthropic API integration (for manual testing)

I've thoroughly tested these changes with both unit tests and manual verification. The updated implementation should now correctly handle various edge cases while maintaining the intended functionality of the caching mechanism.