Refactor SE layer, inverted bottleneck, and MobileNet utilities #120
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This refactors the squeeze and excite (SE) layer and inverted bottleneck layer to make them easier to use with other models. Previously, each MobileNet variant used its own variant of the inverted bottleneck layer. In #113, many of the functions are being reused. So, the motivation for this PR is to make the codepaths shared across these models.
There is now a single
invertedresidual
function that is used by both MobileNet v2 + v3. The activation function can be any function, and the configs/builders for each variant select the specific ones for those models. It also accepts the reduction factor for the SE layer as a keyword. One upside of this that the MobileNet v3 config can now allow a user to specify arbitrary reduction factors (previously it was 4 or no SE).I've moved and renamed
selayer
tosqueeze_excite
. The former is fine for primarily internal use, but now this function is intended to be a reusable block. No functional changes.I also moved and renamed
_make_divisible
to_round_channels
just to make it more clear. Maybe there's a better name, but it is internal.