Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do MMrazor's existing pruning and NAS algorithms support compression of models with Transformer structures? #201

Open
lb-hit opened this issue Jul 19, 2022 · 2 comments
Open

Do MMrazor's existing pruning and NAS algorithms support compression of models with Transformer structures? #201

lb-hit opened this issue Jul 19, 2022 · 2 comments

Comments

@lb-hit
Copy link

lb-hit commented Jul 19, 2022

Checklist

  • I have searched related issues but cannot get the expected help.
  • I have read related documents and don't know what to do.

Describe the question you meet

[here]

Post related information

  1. The output of pip list | grep "mmcv\|mmrazor\|^torch"
    [here]
  2. Your config file if you modified it or created a new one.
[here]
  1. Your train log file if you meet the problem during training.
    [here]
  2. Other code you modified in the mmrazor folder.
    [here]
@wutongshenqiu
Copy link
Contributor

Hi, sibet-lb. Thanks for your attention. Currently there is no pruning or nas algorithms for transformer in MMRazor.

We are working on Autoformer, and it might probably be integrated in MMRazor in a few months.

@lb-hit
Copy link
Author

lb-hit commented Jul 20, 2022

@wutongshenqiu
hi， is there a distillation algorithm suitable for transformer, is MMrazaor's quantization algorithm coming out soon, is there a quantization algorithm suitable for transformer?
I want to compress DETR, any recommendation?

humu789 pushed a commit to humu789/mmrazor that referenced this issue Feb 13, 2023
@grimoire @SingleZombie
@RunningLeon @VVsssssk @AllentDan @hanrui1sensetime
[Refactor] Refactor codebase (open-mmlab#220)
3a785f1 
* [WIP] Refactor v2.0 (open-mmlab#163)

* Refactor backend wrapper

* Refactor mmdet.inference

* Fix

* merge

* refactor utils

* Use deployer and deploy_model to manage pipeline

* Resolve comments

* Add a real inference api function

* rename wrappers

* Set execute to private method

* Rename deployer deploy_model

* Refactor task

* remove type hint

* lint

* Resolve comments

* resolve comments

* lint

* docstring

* [Fix]: Fix bugs in details in refactor branch (open-mmlab#192)

* [WIP] Refactor v2.0 (open-mmlab#163)

* Refactor backend wrapper

* Refactor mmdet.inference

* Fix

* merge

* refactor utils

* Use deployer and deploy_model to manage pipeline

* Resolve comments

* Add a real inference api function

* rename wrappers

* Set execute to private method

* Rename deployer deploy_model

* Refactor task

* remove type hint

* lint

* Resolve comments

* resolve comments

* lint

* docstring

* Fix errors

* lint

* resolve comments

* fix bugs

* conflict

* lint and typo

* Resolve comment

* refactor mmseg (open-mmlab#201)

* support mmseg

* fix docstring

* fix docstring

* [Refactor]: Get the count of backend files (open-mmlab#202)

* Fix backend files

* resolve comments

* lint

* Fix ncnn

* [Refactor]: Refactor folders of mmdet (open-mmlab#200)

* Move folders

* lint

* test object detection model

* lint

* reset changes

* fix openvino

* resolve comments

* __init__.py

* Fix path

* [Refactor]: move mmseg (open-mmlab#206)

* [Refactor]: Refactor mmedit (open-mmlab#205)

* feature mmedit

* edit2.0

* edit

* refactor mmedit

* fix __init__.py

* fix __init__

* fix formai

* fix comment

* fix comment

* Fix wrong func_name of ConvFCBBoxHead (open-mmlab#209)

* [Refactor]: Refactor mmdet unit test (open-mmlab#207)

* Move folders

* lint

* test object detection model

* lint

* WIP

* remove print

* finish unit test

* Fix tests

* resolve comments

* Add mask test

* lint

* resolve comments

* Refine cfg file

* Move files

* add files

* Fix path

* [Unittest]: Refine the unit tests in mmdet open-mmlab#214

* [Refactor] refactor mmocr to mmdeploy/codebase (open-mmlab#213)

* refactor mmocr to mmdeploy/codebase

* fix docstring of show_result

* fix docstring of visualize

* refine docstring

* replace print with logging

* refince codes

* resolve comments

* resolve comments

* [Refactor]: mmseg  tests (open-mmlab#210)

* refactor mmseg tests

* rename test_codebase

* update

* add model.py

* fix

* [Refactor] Refactor mmcls and the package (open-mmlab#217)

* refactor mmcls

* fix yapf

* fix isort

* refactor-mmcls-package

* fix print to logging

* fix docstrings according to others comments

* fix comments

* fix comments

* fix allentdans comment in pr215

* remove mmocr init

* [Refactor] Refactor mmedit tests (open-mmlab#212)

* feature mmedit

* edit2.0

* edit

* refactor mmedit

* fix __init__.py

* fix __init__

* fix formai

* fix comment

* fix comment

* buff

* edit test and code refactor

* refactor dir

* refactor tests/mmedit

* fix docstring

* add test coverage

* fix lint

* fix comment

* fix comment

* Update typehint (open-mmlab#216)

* update type hint

* update docstring

* update

* remove file

* fix ppl

* Refine get_predefined_partition_cfg

* fix tensorrt version > 8

* move parse_cuda_device_id to device.py

* Fix cascade

* onnx2ncnn docstring

Co-authored-by: Yifan Zhou <singlezombie@163.com>
Co-authored-by: RunningLeon <maningsheng@sensetime.com>
Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com>
Co-authored-by: AllentDan <41138331+AllentDan@users.noreply.github.com>
Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wutongshenqiu @lb-hit