Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor SE layer, inverted bottleneck, and MobileNet utilities #120

Merged
merged 4 commits into from
Feb 18, 2022

Conversation

darsnack
Copy link
Member

@darsnack darsnack commented Feb 17, 2022

This refactors the squeeze and excite (SE) layer and inverted bottleneck layer to make them easier to use with other models. Previously, each MobileNet variant used its own variant of the inverted bottleneck layer. In #113, many of the functions are being reused. So, the motivation for this PR is to make the codepaths shared across these models.

There is now a single invertedresidual function that is used by both MobileNet v2 + v3. The activation function can be any function, and the configs/builders for each variant select the specific ones for those models. It also accepts the reduction factor for the SE layer as a keyword. One upside of this that the MobileNet v3 config can now allow a user to specify arbitrary reduction factors (previously it was 4 or no SE).

I've moved and renamed selayer to squeeze_excite. The former is fine for primarily internal use, but now this function is intended to be a reusable block. No functional changes.

I also moved and renamed _make_divisible to _round_channels just to make it more clear. Maybe there's a better name, but it is internal.

@darsnack darsnack requested a review from ToucheSir February 17, 2022 20:06
src/utilities.jl Outdated Show resolved Hide resolved
@darsnack darsnack merged commit dae5b87 into FluxML:master Feb 18, 2022
@darsnack darsnack deleted the invert-bottleneck branch February 18, 2022 18:28
@darsnack darsnack mentioned this pull request Feb 18, 2022
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants