Listening  to ae's universe-shattering   3rd episode  of Special
Education,  while  trying  to just barely  grok my way into deep
learning, this was the series of visions I had. 

Using  Hopfield  networks,  which  are a kind of  neurophysicist
spiking toy model that always converges to a stable state due to
its definition  in terms  of the lagrangian of the neurons  (the
Lagrangian  is an often useful  equation  of Lagrangian  :=  (is
defined as) Kinetic energy - the potential energy 

(Plus whatever lagrange modifier/constraints  which is  probably
important but I didn't see that mentioned yet). 

In the unevolved case, a finished-training stable-state Hopfield
network takes a distorted or incomplete  input of the dimensions
of what it learned from, and returns the best match. Traditional
hopfield networks  have memory for n neurons ~ O ( 0.14 n ), but
a modern modification  gives a storage efficiency of O ( exp n )
(crazy) associated with activation functions like 

(lambda (x) (cond ((< x 0) 0) (t (expt x 2))))

but that's not really here nor there.

With some unlettered  sticking  points, and just with the simple
notion   of  a Hopfield  network,  I was imagining  having   two
hopfield  networks,  one storing truly arbitrary  anaphora  that
have been encountered before 

I : original-form

(progn (get-out pot) (put-on stove pot) (fill-with water pot))

Say we encode (in different hopfield nets)

II : LEAVES.txt

(a (b c) (d e c) (f g c)) as the leaf structure

and separately

III : INPUTS.txt

(progn get-out pot put-on stove fill-with water)

Then we can recover the original form without deep learning like
this: 

(multiple-value-bind (leaves inputs)
 (apply 'values
  (mapcar
   (lambda (x)
    (with-open-file (in (format nil "~a.txt"  x)) (read in)))
   '(leaves inputs)))
 (eval `(let (,@(pairlis '(a b c d e f g) 
                        (mapcar 'list inputs)))
  ,leaves)))

Now the applicability  of a Hopfield  net - a best-matcher  - is
that when faced with a fragment  of a new problem,  a best-match
leaf structure  can be shown  up : and a partially defined  leaf
structure   or  a  partially-known  argument  list can  turn  up
argument   lists.   Metacircularly,  there could be a  class  of
memorised  functions  whose function is to modify a common  leaf
structure   for  another  purpose,   and/or  an  argument   list
similarly. 

A research extension  would be to somehow have search references
into  this hopfield  net supporting  larger equations   of  this
hopfield net. 

I was thinking of using gray level ~ graphic-char-p,  prin1s  of
II and III above in the normalised range 0..1 if that's actually
important, and simply define a fixed character length. 

So far, my impression  was that whereas Bayesian statistics is a
wholly   robotic  tool to be operated  by experts,  for  example
Hopfield  nets are the purview  of physicists  (a physicist   is
anyone who doesn't afraid of calculus). 

This is quite counter to the python module providing C++ company
advertisements  everywhere, which imply that deep learning  is a
flight  dashboard  ready for operation by pilots,  like Bayesian
statistics  is.  That's not my impression  at all.  In the sense
that  Bayesian  statistics  is about a hundred  years old,  deep
learning is about 50-60 years old; maybe that's the difference. 

I'm not sure  how to make it dynamical.  Since  it by definition
converges (on the condition of symmetric edge weights), normally
it is trained  to convergence:  However Hopfield  found that the
chaos introduced by some asymmetries  did not necessarily affect
other useful bits of the network. That makes me hopeful that new
data could be added in a funky way, if somehow a tour through  a
new-data-add  cycle could be provoked  in between  uses, without
blandly retraining the entire network to a different completion.
But  that kind of local  limit cycle  modification  is  probably
badly behaved, in general. 

* I still didn't implement this for the first time myself so WIP