Skip to content

Commit

Permalink
fix(base-grpc): close channel in base grpc server (#3734)
Browse files Browse the repository at this point in the history
If the LLM does not implement any logic for PredictStream, we close the
channel immediately to not leave the process hanging.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
  • Loading branch information
mudler authored Oct 5, 2024
1 parent e28e808 commit 092bb0b
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions pkg/grpc/base/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ func (llm *Base) Predict(opts *pb.PredictOptions) (string, error) {
}

func (llm *Base) PredictStream(opts *pb.PredictOptions, results chan string) error {
close(results)
return fmt.Errorf("unimplemented")
}

Expand Down

0 comments on commit 092bb0b

Please sign in to comment.