Christiansen, M.H. and Chater, N. (1999) Connectionist natural language processing: The state of the art. Cognitive Science, 23, 417-437.
This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models ore likely to have an important influence on future research.
Joanisse, M.F. and Seidenberg, M.S. (1999) Impairments in verb morphology after brain injury: A connectionist model. Proceedings of the National Academy of Sciences of the United States of America, 96, 7592-7597.
The formation of the past tense of verbs in English has been the focus of the debate concerning connectionist vs. symbolic accounts of language. Brain-injured patients differ with respect to whether they are more impaired in generating irregular past tenses (TAKE-TOOK) or past tenses for nonce verbs (WUG-WUGGED). Such dissociations ha, have been taken as evidence for distinct "rule" and "associative" memory systems in morphology and against the connectionist approach in which a single system is used for all forms. We describe a simulation model in which these impairments arise from damage to phonological or semantic information, which have different effects on generalization and irregular forms, respectively. The results pro,ide an account of the bases of impairments in verb morphology and show that these impairments can be explained within connectionist models that do not use rules or a separate mechanism for exceptions.
Marcus, G.F., Vijayan, S., Rao, S.B. and Vishton, P.M. (1999) Rule learning by seven-month-old infants. Science, 283, 77-80.
A fundamental task of language acquisition is to extract abstract algebraic rules. Three experiments show that 7-month- old infants attend longer to sentences with unfamiliar structures than to sentences with familiar structures. The design of the artificial language task used in these experiments ensured that this discrimination could not be performed by counting, by a system that is sensitive only to transitional probabilities, or by a popular class of simple neural network models. Instead, these results suggest that infants can represent, extract, and generalize abstract algebraic rules.
MarslenWilson, W.D. and Tyler, L.K. (1997) Dissociating types of mental computation. Nature, 387, 592-594.
A fundamental issue in the study of cognition and the brain is the nature of mental computation. How far does this depend on internally represented systems of rules, expressed as strings of symbols with a syntax, as opposed to more distributed neural systems, operating subsymbolically and without syntax? The mental representation of the regular and irregular past tense of the English verb has become a crucial test case for this debate. Single-mechanism approaches argue that current multilayer connectionist networks can account for the learning and representation both of regular and of irregular forms(1,2). Dual-mechanism approaches, although accepting connectionist accounts for the irregular forms, argue that a symbolic, rule- based system is required to explain the properties of the regular past tense and, by extension, the properties of language and cognition in general(3-5). We show here that the regular and irregular past tense are supported by different neural systems, which can become dissociated by damage to the brain(6,7). This is evidence for functional and neurological distinctions in the types of mental computation that support these different aspects of linguistic and cognitive performance.
Marslen-Wilson, W. and Tyler, L.K. (1998) Rules, representations, and the English past tense. Trends in Cognitive Sciences, 2, 428-435.
The significance of the English past tense in current cognitive science is that it offers a contrast between a potentially rule-based system - the procedures for forming the past tense - and an unpredictable and idiosyncratic set of irregular forms. This contrast has become a focus for a wide-ranging debate about whether mental computation requires the use of symbols. Highly regular combinatorial phenomena, such as the regular past tense, are prime candidates for rule-based symbolic computation. Earlier research concentrated on the evidence for this during language acquisition, the last five years attention has shifted towards the properties of the adult system, and here some recent research into the neural correlates of the two types of The evidence suggests that there are divergences in the neural systems underlying the generation and perception of regular and irregular forms. Regular to involve primarily combinatorial processes, while irregular forms appear to have a hybrid status, sharing their semantic properties with the regular forms but diverging in the phonological domain, where their form representations are stored at how children learned the English regular and irregular verb systems. Over complete units. This indicates that the regular and irregular past tenses may not, after all, provide a clean contrast in the types of mental computation they implicate.
Plunkett, K. and Juola, P. (1999) A connectionist model of English past tense and plural morphology. Cognitive Science, 23, 463-490.
The acquisition of English noun and verb morphology is modeled using a single-system connectionist network. The network is trained to produce the plurals and past tense forms of a large corpus of monosyllabic English nouns and verbs. The developmental trajectory of network performance is analyzed in detail and is shown to mimic a number of important features of the acquisition of English noun and verb morphology in young children. These include an initial error-free period of performance on both nouns and verbs followed by a period of intermittent over-regularization of irregular nouns and verbs. Errors in the model show evidence of phonological conditioning and frequency effects. Furthermore, the network demonstrates a strong tendency to regularize denominal verbs and deverbal nouns and masters the principles of voicing assimilation. Despite their incorporation into a single-system network, nouns and verbs exhibit some important differences in their profiles of acquisition. Most importantly, noun inflections are acquired earlier than verb inflections. The simulations generate several empirical predictions that can be used to evaluate further the suitability of this type of cognitive architecture in the domain of inflectional morphology.
Polk, TA & Farah, MJ (1998) The neural development and organization of letter recognition: evidence from functional neuroimaging, computational modeling, and behavioral studies. Proceedings of the National Academy of Sciences of the United States of America, Vol.95, No.3, Pp.847-852 .
Although much of the brain's functional organization is genetically predetermined, it appears that some noninnate functions can come to depend on dedicated and segregated neural tissue, In this paper, we describe a series of experiments that have investigated the neural development and organization of one such noninnate function: letter recognition, Functional neuroimaging demonstrates that letter and digit recognition depend on different neural substrates in some literate adults, How could the processing of two stimulus categories that are distinguished solely by cultural conventions become segregated in the brain? One possibility is that correlation-based learning in the brain leads to a spatial organization in cortex that reflects the temporal and spatial clustering of letters with letters in the environment. Simulations confirm that environmental cooccurrence does indeed lead to spatial localization in a neural network that uses correlation- based learning, Furthermore, behavioral studies confirm one critical prediction of this co-occurrence hypothesis, namely, that subjects exposed to a visual environment in which letters and digits occur together rather than separately (postal workers who process letters and digits together in Canadian postal codes) do indeed show less behavioral evidence for segregated letter and digit processing.
Saksida, L.M. (1999) Effects of similarity and experience on discrimination learning: A nonassociative connectionist model of perceptual learning. Journal of Experimental Psychology-Animal Behavior Processes, 25, 308-323.
This article describes a novel connectionist model of perceptual learning (PL) that provides a mechanism for nonassociative differentiation (J. J. Gibson & E. J. Gibson, 1955). The model begins with the assumption that 2 processes-1 that decreases associability and 1 that increases discriminability-operate during preexposure (S. Champ & G. Hall, 1981). In contrast to other models (e.g., I. P. L. McLaren, H. Kaye, & N. J. Mackintosh, 1989), in the current model the mechanisms for these processes are compatible with a configural model of associative learning. A set of simulations demonstrates that the present model can account for critical PL phenomena such as exposure learning and effects of similarity on discrimination. It is also shown that the model can explain the paradoxical result that preexposure to stimuli can either facilitate or impair subsequent discrimination learning. Predictions made by the model are discussed in relation to extant theories of PL.