Hopfield Networks

This repository contains simple implementations of the family of Hopfield Networks discussed in the paper "Hopfield Networks is All You Need" and its associated blog post.

Installation

pip install -r requirements.txt

Classic Hopfield Network

Memory for binary retrieval (see Hopfield, 1982)

python classic_hopfield.py

with output:

Modern Hopfield Network

Improved memory for binary retrieval (see Demircigil et al., 2017)

python modern_hopfield.py

with output:

Modern Hopfield Network with Continuous States

The new network proposed in the paper, equivalent to the attention mechanism used in Transformers (up to linear transformations of its inputs)

python continuous_hopfield.py

with output: