"Hopfield network is all you need" in J (part 1)

We are going to implement the functions proposed in the paper 'Hopfield network is all you need', a fantastic paper that offers an excellent introduction to contemporary Hopfield networks.

General Utility Functions

To facilitate the implementation of the Hopfield network equations, we will write some verbs that will be useful throughout our process.

rndw

rnd=: ?@#&0
rndw=: (+/\%+/)@[ I. rnd@]

This verb is explained here: 

it allows generating random numbers according to a certain distribution. Literally 'weighted random'.

Example:

   100 0 rndw 10
0 0 0 0 0 0 0 0 0 0
   50 50 rndw 10
0 0 0 1 1 1 1 1 0 1

dot

dot =: +/ . *

This is matrix multiplication. More information here:

https://aplwiki.com/wiki/Inner_Product 

https://en.wikipedia.org/wiki/Dot_product (en)  

https://fr.wikipedia.org/wiki/Produit_matriciel (fr)

Example:

  (i.2 3) dot (i. 3 2)
10 13
28 40

Conversion of Binary Data to Bipolar

tobipolar=: [: <: 2 * ]
tobinary =: tobipolar^:_1

At the beginning of the implementation of Hopfield networks, we will use binary data (referred to as patterns in the context of Hopfield networks), composed of 1s and 0s. The canonical Hopfield algorithm uses polar data, where each 0 is replaced by -1. Therefore, it is necessary to define two verbs to switch from one to the other. Note the use of ^:_1 to express the 'negation' of the first verb.

Example:

   tobipolar 0 1 0 0 0 1
_1 1 _1 _1 _1 1
   tobinary _1 1 _1 _1 _1 1
0 1 0 0 0 1

Displaying Patterns

format =: '.#'{~ 0&<

This verb is compatible with both binary and bipolar data. It allows for a slightly more pleasant display of patterns.

Example:

Nletter
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1
0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0
0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 0 0
0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 1 1 0 0
0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0
0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0
0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0
0 0 0 1 1 1 0 1 1 1 1 0 0 0 0 0 1 1 0 0
0 0 0 1 1 1 0 0 1 1 1 1 0 0 0 0 1 1 0 0
0 0 0 1 1 1 0 0 0 0 1 1 1 1 0 0 1 1 0 0
0 0 0 1 1 1 0 0 0 0 1 1 1 1 0 0 1 1 0 0
0 0 0 1 1 1 0 0 0 0 1 1 1 1 1 0 1 1 0 0
0 0 0 1 1 1 0 0 0 0 0 1 1 1 1 1 1 1 0 0
0 0 0 1 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0
0 0 0 1 1 1 0 0 0 0 0 0 0 1 1 1 1 1 0 0
0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0
0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0
0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
   format Nletter
                   
######       #######
 ######        ####
   #####        ##  
   ######       ##  
   #######      ##  
   #######      ##  
   #######      ##  
   ### ####     ##  
   ###  ####    ##  
   ###    ####  ##  
   ###    ####  ##  
   ###    ##### ##  
   ###     #######  
   ###      ######  
   ###       #####  
   ###        ####  
   ###        ####  
  #####        ###  

To be continued

We now have everything we need to implement the equations from the paper 'Hopfield network is all you need', which we will do in a subsequent article.


Commentaires