Variable Binding in Biologically Plausible Neural Networks
Douwe Kiela

Abstract:
One of the most essential aspects of logic is the ability to bind
variables. Without that ability, we cannot represent generic rules in
a logical form. In the past decades there has been a lot of interest
in implementing logic on neural networks, but this was largely
restricted to propositional logic. There have been attempts to
implement more sophisticated logics on neural networks, with mixed
success.  So far, there have been no results that conclusively (and
non-theoretically) show that first-order logic can successfully be
implemented on neural networks. The most important step towards doing
just that, or at least doing so for a fragment, is implementing
variable binding on neural networks.
  In the study of logic and neural networks, the connectionist paradigm
has been of pivotal importance all across the domains of cognitive
science. However, one might consider traditional connectionism
slightly outdated or oversimplified compared to our current knowledge
of the workings of neurons and the brain. Since the advent of the
original artificial neural networks, the discipline of computational
neuroscience has made significant progress. The fine-grained dynamics
in such models may provide new insight into the relative standstill
that the study of logic (using variables) and neural networks has
suffered from in recent years. This interrelation has, so far, been
largely unexplored for these biologically more plausible models.
  The purpose of the thesis at hand is to unite these two points,
insofar as that it aims to show that biologically plausible neural
networks are capable of representing predicates and are capable of
performing variable binding on the predicate’s variables. The results
presented in the current thesis are preliminary, meaning that they are
merely meant to show that exploring the combination between logic and
computational neuroscience can be a fruitful approach and that it
should be pursued much further. The outline of the thesis is as
follows.  First we will establish a solid foundation, covering the
background of the study of logic and neural networks in depth, from
the inception of artificial neurons to the representation of
predicates. In chapter 3, the binding problem–one of the major
hurdles to be taken before variable binding can even be attempted–is
discussed, along with the solutions that have been proposed over the
years.  Chapter 4 makes explicit the distinction between connectionism
and computational neuroscience, in order to draw attention to
biologically plausible neurons.  Chapter 5 briefly describes the
methodology. The correlation between spike times as a measure for
binding is described and the software chosen to perform the
simulations is introduced, as well as the software used to analyze the
results.  Since biologically plausible networks are necessarily highly
parameterized, the chosen parameters are discussed as well. The next
chapter will turn to the actual experiments used to show that such
networks are indeed capable of performing variable binding. Several
criteria have been set in the preceding chapters, which will be
addressed in the models. Chapter 7 will briefly discuss these results
and shed light on some of the remaining issues. Moreover, it discusses
the conception of logic that is at the base of what the thesis
proposes; and discusses some important aspects of extending the
research presented. The closing chapter will look back on what has
been covered in the thesis and provides a brief roadmap towards
achieving the final goal of implementing first-order logic in full.