Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc]fix serving doc link #2490

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ python/build_cpu.sh
python/fastdeploy/ThirdPartyNotices*
*.so*
fpython/astdeploy/libs/third_libs
fastdeploy/core/config.h
fastdeploy/pybind/main.cc
python/fastdeploy/libs/lib*
python/fastdeploy/libs/third_libs
Expand All @@ -29,7 +28,6 @@ python/scripts/process_libraries.py
.DS_Store
miniprogram_npm
node_modules
.DS_Store
dist
etc
lib
Expand All @@ -40,6 +38,7 @@ yalc.*
.yalc
examples/vision/collect_quantize_cc.sh
examples/vision/tests_quantize
ppocr_keys_v1.txt
fastdeploy/LICENSE
fastdeploy/ThirdPartyNotices.txt
FastDeployCSharp.cmake
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/doc/imgs/12.jpg

# x.y.z为镜像版本号,需参照serving文档替换为数字
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
docker run -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
docker run --gpus all -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.5-21.10 bash
docker exec -it -u root fastdeploy bash
```

Expand All @@ -81,7 +81,7 @@ fastdeployserver --model-repository=/ocr_serving/models
```

参数:
- `model-repository`(required): 整套模型streaming_pp_tts存放的路径.
- `model-repository`(required): 整套模型PP-OCRv3存放的路径.
- `http-port`(optional): HTTP服务的端口号. 默认: `8000`. 本示例中未使用该端口.
- `grpc-port`(optional): GRPC服务的端口号. 默认: `8001`.
- `metrics-port`(optional): 服务端指标的端口号. 默认: `8002`. 本示例中未使用该端口.
Expand Down
2 changes: 1 addition & 1 deletion serving/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

FastDeploy builds an end-to-end serving deployment based on [Triton Inference Server](https://github.com/triton-inference-server/server). The underlying backend uses the FastDeploy high-performance Runtime module and integrates the FastDeploy pre- and post-processing modules to achieve end-to-end serving deployment. It can achieve fast deployment with easy-to-use process and excellent performance.

> FastDeploy also provides an easy-to-use Python service deployment method, refer [PaddleSeg deployment example](../examples/vision/segmentation/paddleseg/serving/simple_serving) for its usage.
> FastDeploy also provides an easy-to-use Python service deployment method, refer [PaddleSeg deployment example](../examples/vision/segmentation/paddleseg/semantic_segmentation/serving/simple_serving/README.md) for its usage.

## Prepare the environment

Expand Down
37 changes: 24 additions & 13 deletions serving/README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,48 +4,59 @@

## 简介

FastDeploy基于[Triton Inference Server](https://github.com/triton-inference-server/server)搭建了端到端的服务化部署。底层后端使用FastDeploy高性能Runtime模块,并串联FastDeploy前后处理模块实现端到端的服务化部署。具有快速部署、使用简单、性能卓越的特性。
FastDeploy基于[Triton Inference Server](https://github.com/triton-inference-server/server)
搭建了端到端的服务化部署。底层后端使用FastDeploy高性能Runtime模块,并串联FastDeploy前后处理模块实现端到端的服务化部署。具有快速部署、使用简单、性能卓越的特性。

>
FastDeploy同时还提供了基于Python搭建的服务化部署能力,只需要通过Python即可启动服务,可参考[PaddleSeg部署示例](../examples/vision/segmentation/paddleseg/semantic_segmentation/serving/simple_serving/README_CN.md)
了解其用法。

> FastDeploy同时还提供了基于Python搭建的服务化部署能力,只需要通过Python即可启动服务,可参考[PaddleSeg部署示例](../examples/vision/segmentation/paddleseg/serving/simple_serving)了解其用法。
## 准备环境

### 环境要求

- Linux
- 如果使用GPU镜像, 要求NVIDIA Driver >= 470(如果是旧的Tesla架构GPU,如T4使用的NVIDIA Driver可以是418.40+、440.33+、450.51+、460.27+)
- 如果使用GPU镜像, 要求NVIDIA Driver >= 470(如果是旧的Tesla架构GPU,如T4使用的NVIDIA
Driver可以是418.40+、440.33+、450.51+、460.27+)

### 获取镜像

#### CPU镜像

CPU镜像仅支持Paddle/ONNX模型在CPU上进行服务化部署,支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime

``` shell
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.7-cpu-only-21.10
```

#### GPU镜像

GPU镜像支持Paddle/ONNX模型在GPU/CPU上进行服务化部署,支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime

```
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.7-gpu-cuda11.4-trt8.5-21.10
```

用户也可根据自身需求,参考如下文档自行编译镜像

- [FastDeploy服务化部署镜像编译说明](docs/zh_CN/compile.md)

## 其它文档

- [模型仓库目录说明](docs/zh_CN/model_repository.md) (说明如何准备模型仓库目录)
- [模型配置说明](docs/zh_CN/model_configuration.md) (说明runtime的配置选项)
- [服务化部署示例](docs/zh_CN/demo.md) (服务化部署示例)
- [客户端访问说明](docs/zh_CN/client.md) (客户端访问说明)
- [Serving可视化部署](docs/zh_CN/vdl_management.md) (Serving可视化部署)


### 服务化部署示例

| 任务场景 | 模型 |
|---|-----------------------------------------------------------------------------------|
| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) |
| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) |
| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) |
| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md) |
| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md) |
| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md) |
| OCR | [PaddleOCR/PP-OCRv3](../examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md) |
| 任务场景 | 模型 |
|----------------|-------------------------------------------------------------------------------------------|
| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) |
| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) |
| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) |
| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md) |
| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md) |
| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md) |
| OCR | [PaddleOCR/PP-OCRv3](../examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md) |
Loading