LANGUAGE:
Why Do Kids Say Goed and Brang?

But if language involves the application of rules, why are there so many exceptions? Why is the past tense of take "took" instead of "taked"? Why do we use "stood," not "standed"? "sang," not "singed"? and "went," not "goed"? Within the rule-based framework, explanations of such apparent exceptions follow two approaches: One tries to construct a more complex set of rules (to cover all the cases). The other introduces a second mechanism, a mental lexicon, to handle cases where the usual rules do not apply.

Perhaps, however, the brain stores language in a way that systems of rules and lists of words can only approximate. The rules of language, such as the past tense, need not be explicitly represented. They could instead stem from the operation of more fundamental underlying processes. In the mid-1980s, a group of psychologists, neuroscientists, and computer scientists began to explore the possibility that lawful performance might reflect the operation of a mechanism that never constructed or consulted a rule per se. For example, in the production of past-tense forms of verbs, the mechanism might simply adjust the connections among the neurons involved in forming past tenses when the network encounters the past-tense form of a word.

David Rumelhart and one of us (McClelland) developed this idea using a computer simulation of a simple neural network (1). The model had two sets of neuron-like units, one for representing the base form of a verb, and one for representing its past tense. The model was trained using repeated presentations of present and past-tense pairs. After each presentation, the connections from the units representing the features of the present tense to the units representing the features of the past tense were adjusted by a small amount to allow the active input units to excite the appropriate output units. The model captured the correct use of both regular and exceptional forms, and it exhibited the capacity to generalize. Thus, within a single network, it accounted for many essential aspects of the past-tense phenomena without a rule or a lexicon.

At this point, Steven Pinker entered the debate. Now well known through his popular science writing, Pinker emerged as a prominent psycholinguist on Chomsky's home ground at the Massachusetts Institute of Technology. In an influential article (2), Pinker and Alan Prince drew on their broad knowledge of language to develop a blistering critique of the neural network model. After correctly noting several shortcomings of the specific model used by Rumelhart and McClelland, they went on to claim that no neural network could ever adequately capture the generality and absoluteness of the English past-tense system.

In a later paper (3), however, Pinker acknowledged that the neural network model did have some positive properties. Unlike a standard lexical lookup mechanism, it captured another type of generalization based on patterns found among the exceptions. Using the pattern found in pairs like "sing-sang" and "ring-rang," children sometimes say "brang" for the past tense of "bring." Writing rules to capture the fact that some verbs form their past tense by changing "i" to "a" won't do, said Pinker, because such rules are not fully generative. The pattern only applies to some words with the short vowel "i," and the ones it does apply to share a "family resemblance" structure of the kind that was well captured by the neural net. (They often end in "-ing," but may end in "-im" as in "swim," "-in" as in "begin," or "-ink" as in "drink.") So, Pinker suggested, we use a rule to form regular past tenses, but a mechanism very much like the Rumelhart-McClelland network to formulate irregular past tenses.

Since Pinker proposed these ideas in 1991, he has pursued them in a wide-ranging series of studies. He and his collaborators have analyzed children's spontaneous speech to discover whether the use of the regular past tense develops abruptly or gradually. They have studied adults' acceptance of such sentences as "the slugger flied out to center field" versus "the slugger flew out to center field." They have explored other putative rule-based systems such as the German plural. And they have examined people with brain damage to see if they could find one group lacking the ability to apply the past-tense rule and another group unable to correctly produce exceptions.

In Words and Rules, Pinker offers a very approachable summary of his ideas and the results of his research. Reviewing the history we sketch above, he notes the depth of the challenge that the neural network view poses to conventional rule-based approaches, and he provides many amusing examples to punctuate his arguments. Pinker considers two single-mechanism approaches--using only rules and using only a neural net--and presents his reasons for believing that neither approach will do by itself and that combining positive features from both is the only plausible approach. He introduces simple ideas clearly and complex ideas gradually, so readers without a background in the field will not be left behind. Along the way, he attempts to explain many puzzling and amusing quirks of language, such as the fact that although we tend to describe someone who eats rats as a "rat eater" rather than a "rats eater," we accept both '"mice eater" and "systems analyst." Similarly, Pinker tries to explain why we say "ladies in waiting" and not "lady in waitings," but "major generals" and not "majors general." Overall, the book tells quite an engaging story, one that anyone who has puzzled over the quirks and foibles of language is quite likely to enjoy.

While everyone may agree that Pinker's story is a pleasure to read, it is important to realize that Words and Rules hardly reflects a consensus view. Instead, it represents one of several positions, each championed by a band of ardent adherents. Pinker makes no attempt to hide this; he states his beliefs and preferences, presents the case for his point of view, and discusses what he sees as the shortcomings of the alternatives, particularly the neural-network approach.

Will this book settle the debate? We do not think so. In our view, it suffers from too many serious problems. Pinker sees the evidence from the empirical studies through rule-tinted glasses. For example, he repeatedly refers to abrupt transitions in children's use of the regular past tense, which are consistent with the sudden discovery of a rule. But the relevant data (4) show much more gradual transitions, as expected in the neural network framework. A subtler form of misperception pervades Pinker's discussion of studies that rely on judgments of acceptability of possible past-tense and plural forms. Many "regular" past-tense forms receive intermediate values on a graded acceptability scale. Such graded judgments appear more consistent with the effects of continuous-valued connection strengths (found in a neural net) than with symbolic rules--which are, as Pinker says, "operations that apply fully to any instance of a category."

Nor does Pinker present an explicit model of the processes by which past tenses are generated. There are gaps and inconsistencies in his explanation of how such items are formed. For example, Pinker fails to adequately describe how knowledge of a cluster of similar past-tense forms gives rise to productive generalization (such as the production of "brang" for the past tense of "bring"). He appeals to broad characteristics of the Rumelhart-McClelland model to address such issues, but also to a traditional notion of lexical entries. His presentation does not demonstrate how these concepts can be reconciled or how his proposed system can capture detailed aspects of the phenomena.

Pinker also mistakenly assimilates all neural network models into an older theoretical tradition called associationism, which interprets intelligence as the linkage of "ideas that have been experienced in close succession or that resemble one another." After casting the crucial theoretical issue as a contest between rule-based and associationist approaches, he argues for a dual system that uses both, without fully recognizing how neural network models erase the distinction between them. Pinker dismisses a host of newer neural network models as adding very little beyond the capabilities of the original Rumelhart-McClelland net, consistently underrepresenting their capabilities. Thus, despite forthrightly acknowledging how revolutionary neural network models are, he stops short of conveying their full potential as the basis for the productive use of language.

Pinker's analysis of the formation of the English past tense is reminiscent of the astronomer Tycho Brahe's attempt to come to terms with Copernican theory. Tycho formulated a compromise that captured some of the appeal of the Copernican approach, while maintaining the key Ptolmeic principle of geocentrism. Correspondingly, Pinker has seen some of the appeal of the neural network approach, and he has found ways to incorporate elements of it in a compromise position that maintains the key principle that language knowledge consists (at least in part) of rules. He relies on the properties of neural networks to address problems with the traditional rules-plus-words point of view. His resistance to the core tenets of the neural network framework, however, leaves him--like Tycho--with a compromise that really deals only partially with the challenge posed by the emerging system.

Thus it may be fitting that Words and Rules appears at the end of the old millennium. Written amidst an ongoing research debate, it captures a transitional mode of thought that may be typical of a scientific revolution in progress. Pinker supplies an engaging treatment of an attempt to find a middle ground between two incommensurate theoretical frameworks, and he provides numerous entries to the stream of ongoing research in which the full potential of the newer, neural network approach will eventually become apparent.

References

1. D. E. Rumelhart and J. L. McClelland, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 2 of Psychological and Biological Models, J. L. McClelland, D. E. Rumelhart, the PDP Research Group (MIT Press, Cambridge, MA, 1986), pp. 216-270.
2. S. Pinker and A. Prince, Cognition 28, 73 (1988).
3. S. Pinker, Science 253, 530 (1991).
4. G. F. Marcus et al., Monogr. Soc. Res. Child Dev. 57, 1 (1992).