Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

面版识别添加onnx支持完善 #12068

Merged
merged 2 commits into from
May 9, 2024
Merged

Conversation

heweisheng
Copy link
Contributor

@heweisheng heweisheng commented May 7, 2024

发现面版识别一开onnx就出问题,看了下其他的反馈,还是修复下这个问题好点

issue tracking:

原来情况面版分析使用Onnx第一个报错:

image

第二个报错

image

  • 修复后

image

Copy link

paddle-bot bot commented May 7, 2024

Thanks for your contribution!

@CLAassistant
Copy link

CLAassistant commented May 7, 2024

CLA assistant check
All committers have signed the CLA.

@GreatV
Copy link
Collaborator

GreatV commented May 8, 2024

麻烦把相关issue也贴出来,修复前和修复后的对比也麻烦贴一下。

@heweisheng
Copy link
Contributor Author

麻烦把相关issue也贴出来,修复前和修复后的对比也麻烦贴一下。

#8028

修改前无法正常加载模型,修改后可以正常加载跟使用
具体修改参考的是table里面的onnx写法
参考paddleocr里的逻辑移除了onnx下会下载模型的行为
image

image

@heweisheng
Copy link
Contributor Author

麻烦把相关issue也贴出来,修复前和修复后的对比也麻烦贴一下。

原来情况面版分析使用Onnx第一个报错:
image
第二个报错
image

@GreatV
Copy link
Collaborator

GreatV commented May 8, 2024

麻烦提供一个最小可运行脚本,我测试一下。

@heweisheng
Copy link
Contributor Author

麻烦提供一个最小可运行脚本,我测试一下。
需要使用paddle2onnx将飞浆的模型转换成onnx模型先,然后使用paddleocr命令就可以复现了
paddleocr --image_dir="c:\Users\heweisheng\Desktop\train_5001.jpg" --type=structure --recovery=true --use_onnx=True --layout_model_dir=E:\PyOcr\model\layout\picodet_lcnet_x1_0_fgd_layout_cdla_infer\inference.onnx --rec_model_dir=E:\PyOcr\model\rec\ch\ch_PP-OCRv4_rec_infer\inference.onnx --det_model_dir=E:\PyOcr\model\det\ch\ch_PP-OCRv4_det_infer\inference.onnx --table_model_dir=E:\PyOcr\model\table\ch_ppstructure_mobile_v2.0_SLANet_infer\inference.onnx

修复前:
image
修复后:
image

Copy link
Collaborator

@GreatV GreatV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

测试了一下是可行的

paddleocr --image_dir=ppstructure/docs/table/1.png \
    --type=structure \
    --recovery=true \
    --use_onnx=True \
    --layout_model_dir=inference/picodet_lcnet_x1_0_fgd_layout_infer/best.onnx \
    --rec_model_dir=inference/ch_PP-OCRv4_rec_infer/best.onnx \
    --det_model_dir=inference/ch_PP-OCRv4_det_infer/best.onnx \
    --table_model_dir=inference/ch_ppstructure_mobile_v2.0_SLANet_infer/best.onnx
[2024/05/08 14:17:26] ppocr DEBUG: Namespace(help='==SUPPRESS==', use_gpu=True, use_xpu=False, use_npu=False, use_mlu=False, ir_optim=True, use_tensorrt=False, min_subgraph_size=15, precision='fp32', gpu_mem=500, gpu_id=0, image_dir='ppstructure/docs/table/1.png', page_num=0, det_algorithm='DB', det_model_dir='inference/ch_PP-OCRv4_det_infer/best.onnx', det_limit_side_len=960, det_limit_type='max', det_box_type='quad', det_db_thresh=0.3, det_db_box_thresh=0.6, det_db_unclip_ratio=1.5, max_batch_size=10, use_dilation=False, det_db_score_mode='fast', det_east_score_thresh=0.8, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_sast_score_thresh=0.5, det_sast_nms_thresh=0.2, det_pse_thresh=0, det_pse_box_thresh=0.85, det_pse_min_area=16, det_pse_scale=1, scales=[8, 16, 32], alpha=1.0, beta=1.0, fourier_degree=5, rec_algorithm='SVTR_LCNet', rec_model_dir='inference/ch_PP-OCRv4_rec_infer/best.onnx', rec_image_inverse=True, rec_image_shape='3, 48, 320', rec_batch_num=6, max_text_length=25, rec_char_dict_path='/home/greatx/repos/PaddleOCR/venv/lib/python3.10/site-packages/paddleocr/ppocr/utils/ppocr_keys_v1.txt', use_space_char=True, vis_font_path='./doc/fonts/simfang.ttf', drop_score=0.5, e2e_algorithm='PGNet', e2e_model_dir=None, e2e_limit_side_len=768, e2e_limit_type='max', e2e_pgnet_score_thresh=0.5, e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_pgnet_valid_set='totaltext', e2e_pgnet_mode='fast', use_angle_cls=False, cls_model_dir=None, cls_image_shape='3, 48, 192', label_list=['0', '180'], cls_batch_num=6, cls_thresh=0.9, enable_mkldnn=False, cpu_threads=10, use_pdserving=False, warmup=False, sr_model_dir=None, sr_image_shape='3, 32, 128', sr_batch_num=1, draw_img_save_dir='./inference_results', save_crop_res=False, crop_res_save_dir='./output', use_mp=False, total_process_num=1, process_id=0, benchmark=False, save_log_path='./log_output/', show_log=True, use_onnx=True, return_word_box=False, output='./output', table_max_len=488, table_algorithm='TableAttn', table_model_dir='inference/ch_ppstructure_mobile_v2.0_SLANet_infer/best.onnx', merge_no_span_structure=True, table_char_dict_path='/home/greatx/repos/PaddleOCR/venv/lib/python3.10/site-packages/paddleocr/ppocr/utils/dict/table_structure_dict_ch.txt', layout_model_dir='inference/picodet_lcnet_x1_0_fgd_layout_infer/best.onnx', layout_dict_path='/home/greatx/repos/PaddleOCR/venv/lib/python3.10/site-packages/paddleocr/ppocr/utils/dict/layout_dict/layout_cdla_dict.txt', layout_score_threshold=0.5, layout_nms_threshold=0.5, kie_algorithm='LayoutXLM', ser_model_dir=None, re_model_dir=None, use_visual_backbone=True, ser_dict_path='../train_data/XFUND/class_list_xfun.txt', ocr_order_method=None, mode='structure', image_orientation=False, layout=True, table=True, ocr=True, recovery=True, use_pdf2docx_api=False, invert=False, binarize=False, alphacolor=(255, 255, 255), lang='ch', det=True, rec=True, type='structure', savefile=False, ocr_version='PP-OCRv4', structure_version='PP-StructureV2')
2024-05-08 14:17:26.855753515 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.374'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.855767846 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.371'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.855770335 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.368'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.855772126 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.383'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.855773937 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.362'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.855776729 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.365'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.855778733 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.377'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.855780905 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.380'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946575171 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.374'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946588916 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.371'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946594841 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.368'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946597684 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.383'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946600059 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.362'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946602918 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.365'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946605226 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.377'. It is not used by any node and should be removed from the model.
2024-05-08 14:17:26.946607431 [W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'p2o.helper.constant.380'. It is not used by any node and should be removed from the model.
[2024/05/08 14:17:27] ppocr INFO: **********ppstructure/docs/table/1.png**********
[2024/05/08 14:17:27] ppocr INFO: processing 1/1 page:
[2024/05/08 14:17:27] ppocr DEBUG: dt_boxes num : 223, elapsed : 0.07577061653137207
[2024/05/08 14:17:28] ppocr DEBUG: rec_res num  : 223, elapsed : 1.2030794620513916
[2024/05/08 14:17:28] ppocr DEBUG: dt_boxes num : 22, elapse : 0.022895097732543945
[2024/05/08 14:17:28] ppocr DEBUG: rec_res num  : 22, elapse : 0.12282299995422363
[2024/05/08 14:17:28] ppocr INFO: docx save to ./output/1_ocr.docx
[2024/05/08 14:17:28] ppocr INFO: {'type': 'table', 'bbox': [14, 4, 836, 310], 'img_idx': 0, 'layout': 'single'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'text', 'bbox': [70, 316, 706, 332], 'img_idx': 0, 'layout': 'single'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'text', 'bbox': [69, 342, 342, 359], 'img_idx': 0, 'layout': 'double'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'figure_caption', 'bbox': [14, 360, 402, 711], 'img_idx': 0, 'layout': 'double'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'text', 'bbox': [12, 729, 410, 848], 'img_idx': 0, 'layout': 'double'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'text', 'bbox': [489, 342, 789, 358], 'img_idx': 0, 'layout': 'double'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'figure_caption', 'bbox': [462, 359, 819, 657], 'img_idx': 0, 'layout': 'double'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'title', 'bbox': [444, 705, 564, 724], 'img_idx': 0, 'layout': 'double'}
[2024/05/08 14:17:28] ppocr INFO: {'type': 'text', 'bbox': [444, 751, 841, 848], 'img_idx': 0, 'layout': 'double'}
[2024/05/08 14:17:28] ppocr INFO: result save to ./output

@GreatV GreatV merged commit 5818196 into PaddlePaddle:main May 9, 2024
2 of 3 checks passed
@luotao1
Copy link
Collaborator

luotao1 commented Oct 15, 2024

@heweisheng Thanks for your contribution! You will receive a beautiful PaddlePaddle gift. Please provide your mailing address by filling out the following questionnaire before October 18th.

Looking forward to the future, we will walk further together in the world of open source!
Click Here :https://paddle.wjx.cn/vm/h4On9gJ.aspx#

@luotao1
Copy link
Collaborator

luotao1 commented Nov 6, 2024

hi, @heweisheng

  • 非常感谢你对飞桨的贡献,我们正在运营一个PFCC组织,会通过定期分享技术知识与发布开发者主导任务的形式持续为飞桨做贡献,详情可见 https://github.com/luotao1 主页说明。
  • 如果你对PFCC有兴趣,请发送邮件至 ext_paddle_oss@baidu.com,我们会邀请你加入~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants