refactor: new caller from interfaces & new service from its option

[woorui]

Description

1. A new Service struct has been introduced, which is responsible for handling the main logic. The Service struct has two main methods: GetChatCompletions() and GetInvoke() that handle the main logic.

2. The CallerProvider struct has been removed.

3. The Service struct has an additional method called LoadOrCreateCaller(), which is responsible for managing the creation and caching of the Caller instances.

4. The NewCaller function, which is used to create a new Caller instance, has a new parameter signature: func (source yomo.Source, reducer yomo.StreamFunction, md metadata.M, callTimeout time.Duration) *Caller. This means that when creating a Service instance, you need to inject the way to create the source, reducer, and how to exchange metadata.
   To facilitate this, a new struct called ServiceOption has been defined. The service.LoadOrCreateCaller() method will call the ServiceOption.SourceBuilder(), ServiceOption.ReducerBuilder(), ServiceOption.MetadataExchanger() and then use the returned values as the parameters for the NewCaller function to create the Caller instance.

the ServiceOption struct:

// ServiceOptions is the option for creating service
type ServiceOptions struct {
	// Logger is the logger for the service
	Logger *slog.Logger
	// Tracer is the tracer for the service
	Tracer trace.Tracer
	// CredentialFunc is the function for getting the credential from the request
	CredentialFunc func(r *http.Request) (string, error)
	// CallerCacheSize is the size of the caller's cache
	CallerCacheSize int
	// CallerCacheTTL is the time to live of the callers cache
	CallerCacheTTL time.Duration
	// CallerCallTimeout is the timeout for awaiting the function response.
	CallerCallTimeout time.Duration
	// SourceBuilder should builds an unconnected source.
	SourceBuilder func(zipperAddr, credential string) yomo.Source
	// ReducerBuilder should builds an unconnected reducer.
	ReducerBuilder func(zipperAddr, credential string) yomo.StreamFunction
	// MetadataExchanger exchanges metadata from the credential.
	MetadataExchanger func(credential string) (metadata.M, error)
}

5. Besides,ServiceOptions also allows you to modify the default logger, tracer, and the method to get the credential, among other configuration parameters.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
yomo	⬜️ Ignored (Inspect)	Visit Preview		Aug 16, 2024 7:51am

[codecov]

Codecov Report

Attention: Patch coverage is 79.32584% with 92 lines in your changes missing coverage. Please review.

Project coverage is 79.87%. Comparing base (f438306) to head (e71f4aa).
Report is 1 commits behind head on master.

Files	Patch %	Lines
pkg/bridge/ai/service.go	81.69%	43 Missing and 26 partials ⚠️
pkg/bridge/ai/api_server.go	54.54%	13 Missing and 2 partials ⚠️
pkg/bridge/ai/caller.go	77.14%	6 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #883      +/-   ##
==========================================
+ Coverage   79.15%   79.87%   +0.72%     
==========================================
  Files          48       49       +1     
  Lines        2835     2847      +12     
==========================================
+ Hits         2244     2274      +30     
+ Misses        437      409      -28     
- Partials      154      164      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[venjiang]

👍