Subsymbolic Natural Language Processing
Risto Miikkulainen draws on recent connectionist work in language comprehension to create a model that can understand natural language. Using the DISCERN system as an example, he describes a general approach to building high-level cognitive models from distributed neural networks and shows how the special properties of such networks are useful in modeling human performance. In this approach connectionist networks are not only plausible models of isolated cognitive phenomena, but also sufficient constituents for complete artificial intelligence systems.
Distributed neural networks have been very successful in modeling isolated cognitive phenomena, but complex high-level behavior has been tractable only with symbolic artificial intelligence techniques. Aiming to bridge this gap, Miikkulainen describes DISCERN, a complete natural language processing system implemented entirely at the subsymbolic level. In DISCERN, distributed neural network models of parsing, generating, reasoning, lexical processing, and episodic memory are integrated into a single system that learns to read, paraphrase, and answer questions about stereotypical narratives.
Miikkulainen's work, which includes a comprehensive survey of the connectionist literature related to natural language processing, will prove especially valuable to researchers interested in practical techniques for high-level representation, inferencing, memory modeling, and modular connectionist architectures.
About the Author
Risto Miikkulainen is an Assistant Professor in the Department of Computer Sciences at The University of Texas at Austin.
“Miikkulainen’s work will set a standard for connectionist. AI research on sentence understanding, and it will serve to demonstrate how various problems can be dealt with that have plagued many other approaches.”
—James L. McClelland, Professor of Psychology, Carnegie Mellon University
“This is perhaps the most comprehensive connectionist model of sentence understanding published to date. It will set a standard for future work. Many people are concerned with the problem of scaling up from small demonstration simulations to larger and more comprehensive models; Miikkulainen suggests ways in which this can be done by an imaginative blending of more conventional AI techniques with some of the recent connectionist approaches.”
—Jeffrey Elman, Professor of Cognitive Science, University of California, San Diego