P2P peer-to-peer training of deep learning tensorflow models
Not to be confused with locally distributed training, However presumably a lot can be learned from tf.train.Supervisor etc
In the Golden age of deep learning, baidu and others have shown that training time can scale almost linearly with the number of GPUs. This gives large corporations an advantage over startups in the run for the best A.I. systems ... until now.
Tensorpeers will empower the community to combine their efforts and GPU time into quickly converging wonderful models.
The architecture has to be slightly different from existing 'parameter server' schemes, because of relatively slow Internet connections. However our optimistic guess is that this won't hinder the success of this project: as long as we find any merging scheme, which successfully combines the gained knowledge of two separate runs, we should be fine.
To speed things up, we base this project on python-libtorrent.
MAC:
brew install libtorrent-rasterbar --with-python
LINUX:
apt-get install python-libtorrent or
apt-get install python3-libtorrent
This is a wildly wide open research area, so if you want to make the world a better place (and or need a PhD thesis): Herewith you have full leverage.