Skip to content

Latest commit

 

History

History
58 lines (45 loc) · 3.4 KB

README.md

File metadata and controls

58 lines (45 loc) · 3.4 KB
Raw

Open Source for Spiking-Multiscale-Attention

We have provided training programs and pre trained weights for the DVS128 Gesture, CIFAR10-DVS and N-Caltech101 datasets. The appendix of the article can be accessed here: link

Pre training weights(one GPU can load and run.)

model datasets Models
SMA-VGG Dvs128 Gesture link
SMA-AZO-VGG Dvs128 Gesture link
SMA-VGG CIFAR10-DVS link
SMA-AZO-VGG CIFAR10-DVS link
SMA-VGG N-Caltech101 link
SMA-AZO-VGG N-Caltech101 link

Operating environment

As described in the appendix of the paper, we utilized three devices in our experiments. Device one was dedicated to conducting experiments on the DVS128 Gesture, CIFAR10-DVS, and N-Caltech101 datasets. Device two was allocated for experiments involving the Imagenet-1K dataset using the ResNet18/34 architecture. Lastly, Device three was employed for experiments on the Imagenet-1K dataset using the ResNet104 network.

The specific configurations of these three devices are shown in the table below: image

Regardless of the configuration, the only core libraries used are spikingjelly==0.0.0.0.14, einops, timm and cupy.

For other unimportant configurations, please refer to requirements.txt

Run the DVS128 Gesture

CUDA_VISIBLE_DEVICE="0,1,2,3" python -m torch.distributed.launch --nproc_per_node 4 vgg8_dvs128_SMA.py

Run the CIFAR10-DVS

CUDA_VISIBLE_DEVICE="0,1,2,3" python -m torch.distributed.launch --nproc_per_node 4 vgg8_cifar10dvs_SMA.py

Run the N-Caltech101

CUDA_VISIBLE_DEVICE="0,1,2,3" python -m torch.distributed.launch --nproc_per_node 4 vgg8_NCaltech101_SMA.py

The dataset visualization methods used in this paper have been integrated into the SpikingJelly framework：

Update

A method save_as_pic has been added to save each frame of an individual event as a .png file. Prior to this, spikingjelly only had the method play_frame to save event data as .gif format.

A method save_every_frame_of_an_entire_DVS_dataset has been added that requires only one line of code to save each frame of every sample in an entire DVS dataset as a .png file.

Please Citing

@article{shan2024advancing,
  title={Advancing Spiking Neural Networks towards Multiscale Spatiotemporal Interaction Learning},
  author={Shan, Yimeng and Zhang, Malu and Zhu, Rui-jie and Qiu, Xuerui and Eshraghian, Jason K and Qu, Haicheng},
  journal={arXiv preprint arXiv:2405.13672},
  year={2024}
}