Skip to content

A modern deep learning framework built to accelerate research and development of AI systems

License

Notifications You must be signed in to change notification settings

johnolafenwa/TorchFusion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

79 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TorchFusion

A modern deep learning framework built to accelerate research and development of AI systems.

Based on PyTorch and fully compatible with pure PyTorch and other pytorch packages, TorchFusion provides a comprehensive extensible training framework with trainers that you can easily use to train, evaluate and run inference with your PyTorch models, A GAN framework that greatly simplifies the process of experimenting with Generative Adversarial Networks Goodfellow et al. 2014, with concrete implementations of a number of GAN algorithms, and a number of high level network layers and utilities to help you be more productive in your work.

The framework is highly extensible, so you can easily create your own custom trainers for specific purposes.

New in 2.0

  • Improved Trainer Framework
  • Support for multiple Inputs and Outputs
  • New utilities for loading images, one-hot encoding and more.
  • New Gan Framework with multiple layers of abstraction and implementation of Hinge GANs, GANs with divergence loss, Wasserstein GANs and Relativistic GANs.
  • New GAN Applications with support for spectral normalization, conditional batch normalization, self attention, projection gans and resnet generators and discriminators
  • A wider range of Initializers
  • Enhanced summary function that not only provides you details about number of parameters, layers, input and output sizes but also provides the number of Flops(Multiply-Adds) for every Linear and Convolution layer in your network. Now, you can know the exact computational cost of any CNN architecure with just a single function!!!
  • Visdom and Tensorboard Support
  • Live metrics and loss visualizations, with option to save them permanently
  • Support for persisting logs permanently
  • Easy to use callbacks

Note: This version of torchfusion is well tested and research-ready, the core framework is now complete, Future releases of TorchFusion will include more specialized functions that will cut across multiple domains of deep learning

An AI Commons project https://aicommons.science Developed and Maintained by John Olafenwa and Moses Olafenwa, brothers, creators of ImageAI, Authors of Introduction to Deep Computer Vision and Co-Founders of AICommons Global Limited

Tutorials and Documentation

Visit torchfusion.readthedocs.io for comprehensive tutorials and examples on how to use Torchfusion


Installing TorchFusion

 pip3 install --upgrade torchfusion 

Installing Pytorch

Visit Pytorch.org for instructions on installing pytorch.



MNIST in Five Minutes

from torchfusion.layers import *
from torchfusion.datasets import *
from torchfusion.metrics import *
import torch.nn as nn
import torch.cuda as cuda
from torch.optim import Adam
from torchfusion.learners import StandardLearner

#load dataset
train_loader = mnist_loader(size=28,batch_size=64)
test_loader = mnist_loader(size=28,train=False,batch_size=64)

#define model
model = nn.Sequential(
    Flatten(),
    Linear(784,100),
    Swish(),
    Linear(100,100),
    Swish(),
    Linear(100,100),
    Swish(),
    Linear(100,10)
)

#move to GPU if available
if cuda.is_available():
    model = model.cuda()

#Setup optimizer and loss function
optimizer = Adam(model.parameters())
loss_fn = nn.CrossEntropyLoss()

#Define metrics
train_metrics = [Accuracy()]
test_metrics = [Accuracy()]

#Initiate Learner
learner = StandardLearner(model)

if __name__ == "__main__":
	
    #Print summary of the model
    print(learner.summary((1,28,28)))

    #initiate training
    learner.train(train_loader,train_metrics=train_metrics,optimizer=optimizer,loss_fn=loss_fn,test_loader=test_loader,test_metrics=test_metrics,num_epochs=30,batch_log=False)




GAN in Five Minutes

from torchfusion.gan.learners import *
from torchfusion.gan.applications import StandardGenerator,StandardProjectionDiscriminator
from torch.optim import Adam
from torchfusion.datasets import fashionmnist_loader
import torch.cuda as cuda
import torch.nn as nn

#Define generator and discriminator
G = StandardGenerator(output_size=(1,32,32),latent_size=128)
D = StandardProjectionDiscriminator(input_size=(1,32,32),apply_sigmoid=False)

#Move to GPU if available
if cuda.is_available():
    G = nn.DataParallel(G.cuda())
    D = nn.DataParallel(D.cuda())

#Setup optimizers
g_optim = Adam(G.parameters(),lr=0.0002,betas=(0.5,0.999))
d_optim = Adam(D.parameters(),lr=0.0002,betas=(0.5,0.999))

#Load the dataset
dataset = fashionmnist_loader(size=32,batch_size=64)

#Init learner
learner = RStandardGanLearner(G,D)

if __name__ == "__main__":
    #init training
    learner.train(dataset,gen_optimizer=g_optim,disc_optimizer=d_optim,save_outputs_interval=500,model_dir="./fashion-gan",latent_size=128,num_epochs=50,batch_log=False)

ImageNet Inference

from torchfusion.learners import *
import torch
from torchvision.models.squeezenet import squeezenet1_1
from torchfusion.utils import load_image,decode_imagenet
from torchfusion.datasets import *
INFER_FOLDER  = r"./images"
MODEL_PATH = r"squeezenet.pth"

#load images 
infer_set = pathimages_loader([INFER_FOLDER],size=224,recursive=False)

#init squeezenet
net = squeezenet1_1()

#init learner and load squeezenet model
learner = StandardLearner(net)
learner.load_model(MODEL_PATH)

#function for predicting from a loader
def predict_loader(data_loader):
    predictions = learner.predict(data_loader)
    for pred in predictions:
        pred = torch.softmax(pred,0)
        class_index = torch.argmax(pred)
        class_name = decode_imagenet(class_index)
        confidence = torch.max(pred)
        print("Prediction: {} , Confidence: {} ".format(class_name, confidence))

#function for predict a single image
def predict_image(image_path):
    img = load_image(image_path,target_size=224,mean=0.5,std=0.5)
    img = img.unsqueeze(0)
    pred = learner.predict(img)
    pred = torch.softmax(pred,0)
    class_index = torch.argmax(pred)
    class_name = decode_imagenet(class_index)
    confidence = torch.max(pred)
    print("Prediction: {} , Confidence: {} ".format(class_name, confidence))


if __name__ == "__main__":
    predict_loader(infer_set)
    predict_image(r"sample.jpg")



Contact Developers


John Olafenwa
Email: johnolafenwa@gmail.com
Website: https://john.aicommons.science
Twitter: @johnolafenwa
Medium : @johnolafenwa
Facebook : olafenwajohn

Moses Olafenwa
Email: guymodscientist@gmail.com
Website: https://moses.aicommons.science
Twitter: @OlafenwaMoses
Medium : @guymodscientist
Facebook : moses.olafenwa


Summary of Resnet50 generated by TorchFusion
 Model Summary
Name                      Input Size                Output Size               Parameters                Multiply Adds (Flops)     
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_1                  [1, 3, 224, 224]          [1, 64, 112, 112]         9408                      118013952                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_1             [1, 64, 112, 112]         [1, 64, 112, 112]         128                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_1                    [1, 64, 112, 112]         [1, 64, 112, 112]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
MaxPool2d_1               [1, 64, 112, 112]         [1, 64, 56, 56]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_2                  [1, 64, 56, 56]           [1, 64, 56, 56]           4096                      12845056                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_2             [1, 64, 56, 56]           [1, 64, 56, 56]           128                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_2                    [1, 64, 56, 56]           [1, 64, 56, 56]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_3                  [1, 64, 56, 56]           [1, 64, 56, 56]           36864                     115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_3             [1, 64, 56, 56]           [1, 64, 56, 56]           128                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_3                    [1, 64, 56, 56]           [1, 64, 56, 56]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_4                  [1, 64, 56, 56]           [1, 256, 56, 56]          16384                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_4             [1, 256, 56, 56]          [1, 256, 56, 56]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_5                  [1, 64, 56, 56]           [1, 256, 56, 56]          16384                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_5             [1, 256, 56, 56]          [1, 256, 56, 56]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_4                    [1, 256, 56, 56]          [1, 256, 56, 56]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_1              [1, 64, 56, 56]           [1, 256, 56, 56]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_6                  [1, 256, 56, 56]          [1, 64, 56, 56]           16384                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_6             [1, 64, 56, 56]           [1, 64, 56, 56]           128                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_5                    [1, 64, 56, 56]           [1, 64, 56, 56]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_7                  [1, 64, 56, 56]           [1, 64, 56, 56]           36864                     115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_7             [1, 64, 56, 56]           [1, 64, 56, 56]           128                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_6                    [1, 64, 56, 56]           [1, 64, 56, 56]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_8                  [1, 64, 56, 56]           [1, 256, 56, 56]          16384                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_8             [1, 256, 56, 56]          [1, 256, 56, 56]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_7                    [1, 256, 56, 56]          [1, 256, 56, 56]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_2              [1, 256, 56, 56]          [1, 256, 56, 56]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_9                  [1, 256, 56, 56]          [1, 64, 56, 56]           16384                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_9             [1, 64, 56, 56]           [1, 64, 56, 56]           128                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_8                    [1, 64, 56, 56]           [1, 64, 56, 56]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_10                 [1, 64, 56, 56]           [1, 64, 56, 56]           36864                     115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_10            [1, 64, 56, 56]           [1, 64, 56, 56]           128                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_9                    [1, 64, 56, 56]           [1, 64, 56, 56]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_11                 [1, 64, 56, 56]           [1, 256, 56, 56]          16384                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_11            [1, 256, 56, 56]          [1, 256, 56, 56]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_10                   [1, 256, 56, 56]          [1, 256, 56, 56]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_3              [1, 256, 56, 56]          [1, 256, 56, 56]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_12                 [1, 256, 56, 56]          [1, 128, 56, 56]          32768                     102760448                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_12            [1, 128, 56, 56]          [1, 128, 56, 56]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_11                   [1, 128, 56, 56]          [1, 128, 56, 56]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_13                 [1, 128, 56, 56]          [1, 128, 28, 28]          147456                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_13            [1, 128, 28, 28]          [1, 128, 28, 28]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_12                   [1, 128, 28, 28]          [1, 128, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_14                 [1, 128, 28, 28]          [1, 512, 28, 28]          65536                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_14            [1, 512, 28, 28]          [1, 512, 28, 28]          1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_15                 [1, 256, 56, 56]          [1, 512, 28, 28]          131072                    102760448                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_15            [1, 512, 28, 28]          [1, 512, 28, 28]          1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_13                   [1, 512, 28, 28]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_4              [1, 256, 56, 56]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_16                 [1, 512, 28, 28]          [1, 128, 28, 28]          65536                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_16            [1, 128, 28, 28]          [1, 128, 28, 28]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_14                   [1, 128, 28, 28]          [1, 128, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_17                 [1, 128, 28, 28]          [1, 128, 28, 28]          147456                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_17            [1, 128, 28, 28]          [1, 128, 28, 28]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_15                   [1, 128, 28, 28]          [1, 128, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_18                 [1, 128, 28, 28]          [1, 512, 28, 28]          65536                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_18            [1, 512, 28, 28]          [1, 512, 28, 28]          1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_16                   [1, 512, 28, 28]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_5              [1, 512, 28, 28]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_19                 [1, 512, 28, 28]          [1, 128, 28, 28]          65536                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_19            [1, 128, 28, 28]          [1, 128, 28, 28]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_17                   [1, 128, 28, 28]          [1, 128, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_20                 [1, 128, 28, 28]          [1, 128, 28, 28]          147456                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_20            [1, 128, 28, 28]          [1, 128, 28, 28]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_18                   [1, 128, 28, 28]          [1, 128, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_21                 [1, 128, 28, 28]          [1, 512, 28, 28]          65536                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_21            [1, 512, 28, 28]          [1, 512, 28, 28]          1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_19                   [1, 512, 28, 28]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_6              [1, 512, 28, 28]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_22                 [1, 512, 28, 28]          [1, 128, 28, 28]          65536                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_22            [1, 128, 28, 28]          [1, 128, 28, 28]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_20                   [1, 128, 28, 28]          [1, 128, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_23                 [1, 128, 28, 28]          [1, 128, 28, 28]          147456                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_23            [1, 128, 28, 28]          [1, 128, 28, 28]          256                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_21                   [1, 128, 28, 28]          [1, 128, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_24                 [1, 128, 28, 28]          [1, 512, 28, 28]          65536                     51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_24            [1, 512, 28, 28]          [1, 512, 28, 28]          1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_22                   [1, 512, 28, 28]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_7              [1, 512, 28, 28]          [1, 512, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_25                 [1, 512, 28, 28]          [1, 256, 28, 28]          131072                    102760448                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_25            [1, 256, 28, 28]          [1, 256, 28, 28]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_23                   [1, 256, 28, 28]          [1, 256, 28, 28]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_26                 [1, 256, 28, 28]          [1, 256, 14, 14]          589824                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_26            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_24                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_27                 [1, 256, 14, 14]          [1, 1024, 14, 14]         262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_27            [1, 1024, 14, 14]         [1, 1024, 14, 14]         2048                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_28                 [1, 512, 28, 28]          [1, 1024, 14, 14]         524288                    102760448                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_28            [1, 1024, 14, 14]         [1, 1024, 14, 14]         2048                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_25                   [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_8              [1, 512, 28, 28]          [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_29                 [1, 1024, 14, 14]         [1, 256, 14, 14]          262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_29            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_26                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_30                 [1, 256, 14, 14]          [1, 256, 14, 14]          589824                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_30            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_27                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_31                 [1, 256, 14, 14]          [1, 1024, 14, 14]         262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_31            [1, 1024, 14, 14]         [1, 1024, 14, 14]         2048                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_28                   [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_9              [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_32                 [1, 1024, 14, 14]         [1, 256, 14, 14]          262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_32            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_29                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_33                 [1, 256, 14, 14]          [1, 256, 14, 14]          589824                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_33            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_30                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_34                 [1, 256, 14, 14]          [1, 1024, 14, 14]         262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_34            [1, 1024, 14, 14]         [1, 1024, 14, 14]         2048                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_31                   [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_10             [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_35                 [1, 1024, 14, 14]         [1, 256, 14, 14]          262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_35            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_32                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_36                 [1, 256, 14, 14]          [1, 256, 14, 14]          589824                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_36            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_33                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_37                 [1, 256, 14, 14]          [1, 1024, 14, 14]         262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_37            [1, 1024, 14, 14]         [1, 1024, 14, 14]         2048                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_34                   [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_11             [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_38                 [1, 1024, 14, 14]         [1, 256, 14, 14]          262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_38            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_35                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_39                 [1, 256, 14, 14]          [1, 256, 14, 14]          589824                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_39            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_36                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_40                 [1, 256, 14, 14]          [1, 1024, 14, 14]         262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_40            [1, 1024, 14, 14]         [1, 1024, 14, 14]         2048                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_37                   [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_12             [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_41                 [1, 1024, 14, 14]         [1, 256, 14, 14]          262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_41            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_38                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_42                 [1, 256, 14, 14]          [1, 256, 14, 14]          589824                    115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_42            [1, 256, 14, 14]          [1, 256, 14, 14]          512                       Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_39                   [1, 256, 14, 14]          [1, 256, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_43                 [1, 256, 14, 14]          [1, 1024, 14, 14]         262144                    51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_43            [1, 1024, 14, 14]         [1, 1024, 14, 14]         2048                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_40                   [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_13             [1, 1024, 14, 14]         [1, 1024, 14, 14]         0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_44                 [1, 1024, 14, 14]         [1, 512, 14, 14]          524288                    102760448                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_44            [1, 512, 14, 14]          [1, 512, 14, 14]          1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_41                   [1, 512, 14, 14]          [1, 512, 14, 14]          0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_45                 [1, 512, 14, 14]          [1, 512, 7, 7]            2359296                   115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_45            [1, 512, 7, 7]            [1, 512, 7, 7]            1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_42                   [1, 512, 7, 7]            [1, 512, 7, 7]            0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_46                 [1, 512, 7, 7]            [1, 2048, 7, 7]           1048576                   51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_46            [1, 2048, 7, 7]           [1, 2048, 7, 7]           4096                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_47                 [1, 1024, 14, 14]         [1, 2048, 7, 7]           2097152                   102760448                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_47            [1, 2048, 7, 7]           [1, 2048, 7, 7]           4096                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_43                   [1, 2048, 7, 7]           [1, 2048, 7, 7]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_14             [1, 1024, 14, 14]         [1, 2048, 7, 7]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_48                 [1, 2048, 7, 7]           [1, 512, 7, 7]            1048576                   51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_48            [1, 512, 7, 7]            [1, 512, 7, 7]            1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_44                   [1, 512, 7, 7]            [1, 512, 7, 7]            0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_49                 [1, 512, 7, 7]            [1, 512, 7, 7]            2359296                   115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_49            [1, 512, 7, 7]            [1, 512, 7, 7]            1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_45                   [1, 512, 7, 7]            [1, 512, 7, 7]            0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_50                 [1, 512, 7, 7]            [1, 2048, 7, 7]           1048576                   51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_50            [1, 2048, 7, 7]           [1, 2048, 7, 7]           4096                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_46                   [1, 2048, 7, 7]           [1, 2048, 7, 7]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_15             [1, 2048, 7, 7]           [1, 2048, 7, 7]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_51                 [1, 2048, 7, 7]           [1, 512, 7, 7]            1048576                   51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_51            [1, 512, 7, 7]            [1, 512, 7, 7]            1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_47                   [1, 512, 7, 7]            [1, 512, 7, 7]            0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_52                 [1, 512, 7, 7]            [1, 512, 7, 7]            2359296                   115605504                 
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_52            [1, 512, 7, 7]            [1, 512, 7, 7]            1024                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_48                   [1, 512, 7, 7]            [1, 512, 7, 7]            0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_53                 [1, 512, 7, 7]            [1, 2048, 7, 7]           1048576                   51380224                  
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_53            [1, 2048, 7, 7]           [1, 2048, 7, 7]           4096                      Not Available             
----------------------------------------------------------------------------------------------------------------------------------
ReLU_49                   [1, 2048, 7, 7]           [1, 2048, 7, 7]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Bottleneck_16             [1, 2048, 7, 7]           [1, 2048, 7, 7]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
AvgPool2d_1               [1, 2048, 7, 7]           [1, 2048, 1, 1]           0                         Not Available             
----------------------------------------------------------------------------------------------------------------------------------
Linear_1                  [1, 2048]                 [1, 1000]                 2049000                   2048000                   
----------------------------------------------------------------------------------------------------------------------------------

Total Parameters: 25557032
----------------------------------------------------------------------------------------------------------------------------------
Total Multiply Adds (For Convolution and Linear Layers only): 4089184256
----------------------------------------------------------------------------------------------------------------------------------
Number of Layers
Conv2d : 53 layers   Bottleneck : 16 layers   MaxPool2d : 1 layers   Linear : 1 layers   BatchNorm2d : 53 layers   AvgPool2d : 1 layers   ReLU : 49 layers