TypeError: __init__() got an unexpected keyword argument 'global_pool'

[aribornstein]

🐛 Bug

Flash backbones is hardcoding the global_pool argument but this isn't relevant for Visual Transformer backbones and causes them to crash

TypeError: __init__() got an unexpected keyword argument 'global_pool'

To Reproduce

model = ImageClassifier(backbone="vit_small_patch16_224", num_classes=10, serializer=Labels())

Expected behavior

global_pool should only be passed if expected model backbone should initialize and not crash

Environment

• PyTorch Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context

[ethanwharris]

Closed by #377