Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support deepseek ai model #989

Merged
merged 2 commits into from
May 23, 2024
Merged

feat: support deepseek ai model #989

merged 2 commits into from
May 23, 2024

Conversation

goooogoooo
Copy link
Contributor

Ⅰ. Describe what this PR did

Support DeepSeek ai model, api documentation: https://platform.deepseek.com/api-docs/zh-cn/

Ⅱ. Does this pull request fix one issue?

fixes #961

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

envoy.yaml:

admin:
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9901
static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address:
          protocol: TCP
          address: 0.0.0.0
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                scheme_header_transformation:
                  scheme_to_overwrite: https
                stat_prefix: ingress_http
                # Output envoy logs to stdout
                access_log:
                  - name: envoy.access_loggers.stdout
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                # Modify as required
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: [ "*" ]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: deepseek
                            timeout: 300s
                http_filters:
                  - name: wasmtest
                    typed_config:
                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      value:
                        config:
                          name: wasmtest
                          vm_config:
                            runtime: envoy.wasm.runtime.v8
                            code:
                              local:
                                filename: /etc/envoy/plugin.wasm
                          configuration:
                            "@type": "type.googleapis.com/google.protobuf.StringValue"
                            value: |
                              {
                                "provider": {
                                  "type": "deepseek",
                                  "apiTokens": [
                                    "******"
                                  ]
                                }
                              }
                  - name: envoy.filters.http.router
  clusters:
    - name: httpbin
      connect_timeout: 30s
      type: LOGICAL_DNS
      # Comment out the following line to test on v6 networks
      dns_lookup_family: V4_ONLY
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: httpbin
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: httpbin
                      port_value: 80
    - name: deepseek
      connect_timeout: 30s
      type: LOGICAL_DNS
      dns_lookup_family: V4_ONLY
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: deepseek
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: api.deepseek.com
                      port_value: 443
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          "sni": "api.deepseek.com"
  • Use deepseek-chat model

Request:

curl "http://localhost:10000/v1/chat/completions"  -H "Content-Type: application/json" -d '{
  "model": "deepseek-chat",            
  "messages": [                                                     
    {                        
      "role": "user",                
      "content": "你好,你是谁?"
    }                                                                                 
  ]                                                 
}'

Response:

{
	"id": "b8720950-569d-4413-b2cf-436e2729db00",
	"choices": [{
		"index": 0,
		"message": {
			"content": "你好!我是一个人工智能助手,专门设计来回答问题、提供信息和帮助解决问题。如果你有任何疑问或需要帮助,请随时告诉我。",
			"role": "assistant"
		},
		"finish_reason": "stop",
		"logprobs": null
	}],
	"created": 1716370331,
	"model": "deepseek-chat",
	"system_fingerprint": "fp_1d4b677f33",
	"object": "chat.completion",
	"usage": {
		"prompt_tokens": 13,
		"completion_tokens": 31,
		"total_tokens": 44
	}
}
  • Use deepseek-coder deepseek-coder

Request:

curl "http://localhost:10000/v1/chat/completions"  -H "Content-Type: application/json" -d '{
  "deepseek-coder": "deepseek-coder",            
  "messages": [                                                     
    {                        
      "role": "user",                
      "content": "你好,你是谁?"
    }                                                                                 
  ]                                                 
}'

Response:

{
	"id": "f3b071c1-e1ad-4590-b0d7-afb1eb1bf529",
	"choices": [{
		"index": 0,
		"message": {
			"content": "你好!我是DeepSeek Coder,由中国的深度求索(DeepSeek)公司开发的编程智能助手。我专门用来回答计算机科学相关的问题。如果你有任何编程或计算机科学相关的问题,欢迎随时向我提问。",
			"role": "assistant"
		},
		"finish_reason": "stop",
		"logprobs": null
	}],
	"created": 1716370341,
	"model": "deepseek-coder",
	"system_fingerprint": "fp_ded2115e5a",
	"object": "chat.completion",
	"usage": {
		"prompt_tokens": 18,
		"completion_tokens": 62,
		"total_tokens": 80
	}
}

Ⅴ. Special notes for reviews

@CLAassistant
Copy link

CLAassistant commented May 22, 2024

CLA assistant check
All committers have signed the CLA.

Copy link
Collaborator

@CH3CHO CH3CHO left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

@CH3CHO CH3CHO merged commit 10f1adc into alibaba:main May 23, 2024
11 checks passed
@goooogoooo goooogoooo deleted the deepseek branch May 23, 2024 01:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AI 代理 Wasm 插件对接 DeepSeek
3 participants