For code samples, please take a look at a couple of files from the TO-BE-PUBLISHED-SOON Ai-Memory Research above
I replicated Deepmind’s Multimodal Few-Shot Learning with Frozen Language Models.
I implemented it using GPT-J as the language model and CLIP’s Vision Transformer as the visual encoder. It uses approximately 6.1 Billion Parameters. Using pre-trained models and preemptible TPUv3-8 enabled me to bring down the training cost to a palatable couple of thousand dollars.
Question 1: In the picture, what color jerseys are the players wearing? Answer: Green tshirts and black shorts. Question 2: What are the players doing? Answer:
Question 1: In the picture, what color jerseys are the players wearing? Answer: Green tshirts and black shorts. Question 2: What are the players doing? Answer: They’re warming up for a game
I designed a novel deep generative 3D-CNN-Transformer Hybrid architecture to build a computationally tractable global climate forecasting engine.
I proposed a novel convolutional architecture that improves upon traditional CNNs by building upon Jeff Hawkins’ brilliant thousand brains theory of intelligence.
I presented a soft proof of why convolutions can be used as a foundation for cortical mini-columns. This raises the question that whether the success of CNNs can be used as weak empirical proof for the 1000 brains theory.
AIM, which stands for AI Memory, is a novel method to attach memory to boost the performance of almost all neural networks.
It takes only a few additional lines of code to implement and significantly improves a model's ability to learn and generalize. The proof of concept demonstrated that AIM has the potential to improve the performance of every single neural network in the world.