The PDP Interactive Handbook

Hello and welcome to the PDP Interactive Primer. This document is intended to provide one with an interactive introduction to the world of Parallel Distributed Processing.


In 1986, Jay McClelland and David Rumelhart published a book entitled Parallel Distributed Processing: Explorations in the Microstructure of Cognition(1986). In the summer of 1994, I was privileged enough to work with Dr. McClelland, and this document is my effort at putting the essentials of the PDP paradigm into an interactive format. I freely admit at the outset that this is an interactive primer for their work, and as such, contains much of the same terminology and ideas, albeit in a simpler, bite-sized form.

I also admit that this is purely my take on their work, and could contain errors. Truely interested parties are encouraged to consult McClelland and Rumelhart(1986)'s work for the definitive word.

Table of Contents



In 1986, James L. McClelland, David E. Rumelhart and the PDP Research Group published a volume which was to become a seminal work in the field of cognitive psychology. Along with several other works, McClelland and Rumelhart's Parallel Distributed Processing: Explorations in the Microstructure of Cognition (1986) represented a break with the conventional cognitive theories of the time (Minsky, 1969; Schank, 1980) and necessitated a paradigm shift, the effects of which are still being felt today. (Allman, 1989)

So just what is Parallel Distributed Processing (hereafter referred to as PDP) all about? Quite simply, proponents of PDP assert that the brain is NOT a computer, not a serial one anyway. In essence, thought is a parallel process, a network of multiple, graded constraints being considered simultaneously. Thought is not a single path of constraints being considered one at a time, as in conventional cognitive models. Moreover, structural differences in the network of constraints are important to the implementation of the thought process and can lead to qualitative differences in the final result. This is in direct opposition to previous cognitive theories which assert that structure has no effect on the outcome of the thought process. (Rumelhart & McClelland, 1986)


However, it is not my intention to argue for or against the validity of PDP. I merely seek to demonstrate that PDP represents a new paradigm from which new experimental methodologies, results, and theories about the nature of human thought can, and have been, derived (Rumelhart & Norman, 1982; Hinton, 1984; McClelland & Rumelhart, 1981). With this in mind, it is vital that students become aware of, and are educated in, the intricacies of this shifting paradigm. Unfortunately, up until this point no primer for PDP existed and certainly not one that would be accessible to individuals on-line through the INTERNET.

This is such a primer, written in hypertext, able to be viewed by hypertext browser, such as Netscape. (In fact, this Primer was written specifically for Netscape, and the results seen on any other browser could be less than perfect: sorry.) This primer covers the basics of PDP, the different kinds of models, and includes some working examples of models in action. While this project is primarily intended for psychology students looking to get a leg up on PDP, before delving into it in detail, its implementation in hypertext virtually assures that other interested individuals all around the world may benefit.

Organization of the Primer

This primer is organized into four main sections:

The first section provides the general framework for PDP used by McClelland and Rumelhart in their book, and subsequently sketches some of the underlying math.

Sections two and three provide an outline of two major types of neural networks: Interactive Activation and Competition Networks and Pattern Associator Networks.

The last section provides a discussion of the interface between Neural Networks and Chaos Theory.

Onward to the first section: The General Framework for PDP.

Back to Table of Contents.

Last Modified: Sept 11,1995.

George Hollich
Temple University