Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add weight quantization in post_training_quanzitaion, test=develop #22445

Merged
merged 5 commits into from
Feb 7, 2020
Merged

Add weight quantization in post_training_quanzitaion, test=develop #22445

merged 5 commits into from
Feb 7, 2020

Conversation

juncaipeng
Copy link
Contributor

In order to reduce the size of model, this api quantizes the weight of some ops from float32 to int8/16. In the inference stage, the quantized weight will be dequantized to float32 again.

@juncaipeng juncaipeng mentioned this pull request Feb 4, 2020
Merged
wzzju
wzzju previously approved these changes Feb 5, 2020
View reviewed changes
Copy link
Contributor

@wzzju wzzju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

python/paddle/fluid/contrib/slim/quantization/post_training_quantization.py Outdated
weight, and it should be 8 or 16. Default is 8.
threshold_rate(float, optional): This api uses abs_max methd to
quantize the weight from float32 to int8/16, and the abs max
value is important for quantization diff. When the is far
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the is far?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@juncaipeng juncaipeng closed this Feb 5, 2020
@juncaipeng juncaipeng reopened this Feb 5, 2020
@juncaipeng juncaipeng closed this Feb 5, 2020
@juncaipeng juncaipeng reopened this Feb 5, 2020
@juncaipeng juncaipeng closed this Feb 5, 2020
@juncaipeng juncaipeng reopened this Feb 5, 2020
wzzju
wzzju approved these changes Feb 6, 2020
View reviewed changes
Copy link
Contributor

@wzzju wzzju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@juncaipeng juncaipeng merged commit 197913e into PaddlePaddle:develop Feb 7, 2020
juncaipeng added a commit to juncaipeng/Paddle that referenced this pull request Feb 7, 2020
@juncaipeng
Add weight quantization in post_training_quanzitaion (PaddlePaddle#22445
2dca4ca 
)

* support weight quantization in post_training_quanzitaion, test=develop
* add test for weight quantization, test=develop
@tangtang586
Copy link

this op name mentioned in the title should be post_training_quantization.

juncaipeng added a commit that referenced this pull request Feb 11, 2020
@juncaipeng @zhiqiu
[cherry-pick] Add weight quantization in post_training_quanzitaion (#…
ad2813b 
…22445) (#22493)

* Add weight quantization in post_training_quanzitaion (#22445)

* [cherry-pick]Support int16 for Tensor (#22423)

* add int16 support, test=develop, test=release/1.7

Co-authored-by: Leo Chen <chenqiuliang@baidu.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wzzju wzzju wzzju approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@juncaipeng @tangtang586 @wzzju