Default VGG dropout probability changed to zero

[adrhill]

PR #270 inadvertently changed the default Dropout probability of VGG models from 0.5 to 0.0:

julia> VGG(11)
VGG(
  Chain(
    Chain(
      Conv((3, 3), 3 => 64, relu, pad=1),  # 1_792 parameters
      MaxPool((2, 2)),
      Conv((3, 3), 64 => 128, relu, pad=1),  # 73_856 parameters
      MaxPool((2, 2)),
      Conv((3, 3), 128 => 256, relu, pad=1),  # 295_168 parameters
      Conv((3, 3), 256 => 256, relu, pad=1),  # 590_080 parameters
      MaxPool((2, 2)),
      Conv((3, 3), 256 => 512, relu, pad=1),  # 1_180_160 parameters
      Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
      MaxPool((2, 2)),
      Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
      Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
      MaxPool((2, 2)),
    ),
    Chain(
      MLUtils.flatten,
      Dense(25088 => 4096, relu),       # 102_764_544 parameters
      Dropout(0.0),                                                    <==
      Dense(4096 => 4096, relu),        # 16_781_312 parameters
      Dropout(0.0),                                                    <==
      Dense(4096 => 1000),              # 4_097_000 parameters
    ),
  ),
)                   # Total: 22 arrays, 132_863_336 parameters, 506.839 MiB.

There currently also isn't any direct way to create a pretrained VGG model with custom dropout_prob:

julia> VGG(11; pretrain=true, dropout_prob=0.5)
ERROR: MethodError: no method matching VGG(::Int64; pretrain::Bool, dropout_prob::Float64)

[theabhirath]

Yep, I missed that when making the VGG to vgg swap 🤦🏽‍♂️ Nice catch, thank you!

[theabhirath]

There currently also isn't any direct way to create a pretrained VGG model with custom dropout_prob.

Oh, this is interesting! I think this can be handled with loadpretrain! in Metalhead if you pass in the model created by vgg and the artifact name. We don't really list the artifacts in the docs (we really should, good change to make in the future) but right now you can just check the Artifacts.toml file – for the VGG models they're just named "vggX".

Only thing to keep in mind is that you'll have to wrap the layers around the model struct because that's what the function expects. Something like:

layers = Metalhead.vgg((224, 224); config = Metalhead.VGG_CONFIGS[11], batchnorm = false, inchannels = 3, nclasses = 1000, dropout_prob = 0.2)
model = VGG(layers)
Metalhead.loadpretrain!(VGG(layers), "vgg11")

should work in for what you seem to have in mind.

[adrhill]

That's good to know, thanks!