Has anyone successfully added other nets like AlexNet into Timm?

[Calvin-Pang]

Hello, I am trying to add AlexNet into timm and use it to solve a problem about image classfication (num_class=3), and I use the AlexNet code from https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py and modify it a bit to let thie model can be created successfully. However, in the processing of training, I found the loss keeps in a high level and unable to train successfully.

And here is my model code alexnet.py:
`import torch
import torch.utils.model_zoo as model_zoo
import torch.nn as nn
from timm.data import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD
from .helpers import build_model_with_cfg
from .registry import register_model
from typing import Any

all = ['AlexNet','alexnet']

def _cfg(url=''):
return {
'url': url, 'num_classes': 1000, 'input_size': (3, 224, 224), 'pool_size': (7, 7),
'crop_pct': 0.875, 'interpolation': 'bicubic',
'mean': IMAGENET_DEFAULT_MEAN, 'std': IMAGENET_DEFAULT_STD,
'first_conv': 'features.0', 'classifier': 'classifier'
}

default_cfgs = {
'alexnet': 'file:///E:/大学资料/HuoLab/pytorch-image-models/alexnet-new-calvin.pth',
}

class AlexNet(nn.Module):
def init(self, num_classes = 1000,in_chans = 3,drop_rate=0):
self.num_classes = num_classes
self.drop_rate = drop_rate
super(AlexNet, self).init()
self.features = nn.Sequential(
nn.Conv2d(in_chans, 64, kernel_size=11, stride=4, padding=2),
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3, stride=2,padding=0),
nn.Conv2d(64, 192, kernel_size=5,stride=1,padding=2),
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3, stride=2,padding=0),
nn.Conv2d(192, 384, kernel_size=3,stride=1,padding=1),
nn.ReLU(inplace=True),
nn.Conv2d(384, 256, kernel_size=3,stride=1,padding=1),
nn.ReLU(inplace=True),
nn.Conv2d(256, 256, kernel_size=3,stride=1,padding=1),
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3, stride=2,padding=0),
)
self.avgpool = nn.AdaptiveAvgPool2d((6, 6))
self.classifier = nn.Sequential(
nn.Dropout(p=0.5),
nn.Linear(256 * 6 * 6, 4096),
nn.ReLU(inplace=True),
nn.Dropout(p=0.5),
nn.Linear(4096, 4096),
nn.ReLU(inplace=True),
nn.Linear(4096, num_classes),
)

def forward(self, x):
    x = self.features(x)
    # x = self.avgpool(x)
    # x = torch.flatten(x, 1)
    x = x.view(x.size(0), 256 * 6 * 6)
    x = self.classifier(x)
    return x

@register_model
def alexnet(pretrained=False, **kwargs):
model = AlexNet(**kwargs)
if pretrained:
model.load_state_dict(model_zoo.load_url(default_cfgs['alexnet']))
return model`

[Calvin-Pang]

[Calvin-Pang]

Sorry again, I found the function still cannot solve my problem. And can anybody help me and find what's wrong is in my code?

[alexander-soare]

@Calvin-Pang using your code with weights from https://download.pytorch.org/models/alexnet-owt-7be5be79.pth I was able to evaluate imagenet val with 56.2% top-1, and then I was able to train on imagenette2-320 (fast ai mini version of imagenet) with

python train.py /home/alexander/Datasets/imagenette2-320 --model alexnet -b 512 --warmup-epochs 1 --pretrained --lr 3e-4 --opt adam --epochs 30

And get to 95% top-1 in 10 epochs.

Maybe check if you can get similar results as a debugging step.

[Calvin-Pang]

OK,thank you very much for your reply! It's the problem of the optimizer and I made it with your method.