Project Description

Background: As we are comprehending language, we develop interpretations without waiting until the ends of words, phrases or sentences. This process requires predictive and continuous coordination of different types of linguistic as well as non-linguistic information. Recent studies have proposed that this information coordination mechanism is further modulated according to the comprehender's assumptions about the speaker. For instance, when comprehenders believe that the speaker has an impairment that causes language and social problems, they may expect the linguistic input to be less reliable and adjust their comprehension behavior accordingly. This line of research uncovers how the bottom-up linguistic input and the top-down expectations and assumptions may constrain each other to guide real-time language processing.

PI: Kurumada (Tanenhaus)

Paper: Grodner, D. J., & Sedivy, J. C. (2011). The effect of speaker-specific information on pragmatic inferences. In E. Gibson., & N. Pearlmutter (Eds.) The processing and acquisition of reference. pp. 239-272.

Study: Experiment 1: Suspension of predictive eye-movements based on pronominal adjectives

Summary: Subjects are randomly assigned to one of two conditions and eye-tracked as they follow verbal instructions to manipulate objects on a computer screen. Target instructions contain a pre-nominal adjective (e.g., "Click on a tall glass"), which usually triggers anticipatory eye movements to a target object (e.g., a tall glass) when a contrasting object (e.g., a small glass) is also present. However, when subjects are told that the speaker has an impairment that prevents her from making logical uses of prenominal adjectives, such anticipatory eye movements were no longer observed. The results suggest that subjects' real-time language comprehension behavior is modulated in a speaker-specific fashion according to their expectations about the utility of linguistic signals.

Impact: This is a pioneering study that opened up a number of avenues of research questions regarding the rationality and automaticity of pragmatic processing in real-time sentence comprehension. This is, however, the only study thus far that showed effects of top-down information about the speaker on the time-course of real-time eye movements. It is yet to be established whether the results are reliably replicable.

Skills Required

Simple stimulus/response programming, R

UR Expo Poster (PDF)

Project Description

Background: Research has demonstrated that native listeners perceptually adapt to systematic properties of non-native speech. In particular, exposure to a wide range of different foreign accents seem to result in perceptual learning of speech that is generalizable to novel utterances, novel talkers and even a novel, unfamiliar accent. The underlying mechanism by which listeners are able to achieve such great flexibility in speech comprehension is not clearly understood. It is unknown whether listener have systematically adjusted the mapping from speech sounds to linguistic representations, or rather, there is a change in their tolerance of what sounds are acceptable word forms. Additionally, it is unclear how properties of specific accents modulate generalization. Results from the study will help us understand the perceptual reorganization that listeners experience and how they cope with speech variability in a multi-linguistic society.

PI: Jaeger (w/ Xin Xie)

Paper: Baese-Berk, M. M., Bradlow, A. R., & Wright, B. A. (2013). Accent-independent adaptation to foreign accented speech. The Journal of the Acoustical Society of America, 133, EL174-EL180.

Study: Experiment 1: Improvement in recognition accuracy of foreign-accented speech sentences

Summary: Participants are instructed to listen to foreign-accented English sentences and transcribe every sentence. Transcription accuracy is measured by the percentage of keywords correctly recognized. The sentences are produced by five speakers, each with a different foreign accent (multiple-accent training group). Sentence presentation is blocked by speaker and no feedback is given. After two training sessions, participants complete two post-tests. The post-tests require transcription of sentences from two novel non-native speakers of English: one with a trained accent and one with an untrained accent. Results are compared against those from two other groups (single-accent training group and no foreign-accent group). During training, the single-accent training group hear five speakers of the same accent (the accent of the 'trained accent' speaker at test); the no foreign-accent group hear five native-English speakers. Results are that both single-accent training and multiple-accent training are effective to improve recognition of the trained accent; however, only multiple-accent training promotes generalization to an untrained accent.

Impact: This is a study that has been widely discussed in the literature (35 citations since 2013). In particular, it is taken to suggest that exposure to multiple accents elicits accent-independent adaptation in speech processing. This suggestion has important implications for models of speech perceptual learning. Replication of this study is needed in order to further investigate the perceptual and cognitive processes that support such adaptation.

Skills Required

Simple stimulus/response programming, ANOVA

Resulting conference presentations and papers

• Lee,​ ​C​., Oey, L., Simon, E., Xie, X., Jaeger, T. F. (2017). An investigation into audio perception studies on Amazon Mechanical Turk. Poster presented at the 23rd Architectures and Mechanisms of Language Processing Conference, Lancaster, UK.
• Oey, L., Lee,​ ​C​., Simon, E., Xie, X., Jaeger, T. F. (2017). How we comprehend foreign-accented speech: Learning to generalize across talkers. Poster presented at the 23rd Architectures and Mechanisms of Language Processing Conference, Lancaster, UK.
• Lee,​ ​C​., Oey, L., Simon, E., Xie, X., Jaeger, T. F. (2017). How we comprehend foreign-accented speech: Learning to generalize across talkers. Poster accepted at the 31st National Conference on Undergraduate Research, Memphis, TN.
• Lee, C., Oey, L., Simon, E., Xie, X., Jaeger, T. F. (submitted). Limits in cross-talker generalization of adaptation to foreign-accented speech.

UR Expo Poster (PDF)

Project Description

Background: Efforts to understand the relationship between human intelligence and perceptual processing have a long history. Despite scientific endeavor, the reported links between perception and intelligence, while well established, are modest, with uncorrected correlations typically between 0.2-0.4. On average, high IQ individuals tend to have better sensory discriminations and are faster at processing sensory stimuli. This project aims to elucidate mechanisms that underlie a recent discovery of a strong and seemingly counterintuitive relationship between intelligence and visual motion perception (Melnick et al., 2013). In this link, highly intelligent individuals, while excelling at perceiving small moving objects, exhibit disproportionate difficulties in perceiving the motion of large stimuli (r > 0.7). What mechanisms account for this link is unknown. The key insight is that weak spatial suppression in low IQ individuals can be explained by increased internal noise. At high internal noise, the brain benefits from averaging processes (Faisal, Selen & Wolpert, 2008). This enhances perception of normally suppressed, large moving stimuli—a result exhibited by low IQ individuals. On the other hand, low internal noise permits operation of functionally advantageous suppressive processes that impair motion perception of large stimuli—a perceptual feature associated with high IQ.

PI: Tadin

Paper: Melnick, M.D., Harrison, B.R., Park, S., Bennetto, L. & Tadin, D. (2013) A strong interactive link between sensory discriminations and intelligence. Current Biology, 23, 1013-1017

Study: Links among internal noise, spatial suppression and intelligence

Summary: The main hypothesis in this proposal is that individual differences in internal noise account for this strong link between intelligence and perception. This hypothesis, motivated by ubiquitous and inescapable roles of noise in neural processing (Faisal, Selen & Wolpert, 2008), will be tested by measuring intelligence (using Raven's progressive matrices), perceptual suppression and internal noise.

Impact: Despite the paramount importance of internal noise on brain function, the proposed link between internal noise and intelligence is surprisingly unexplored. Thus, this project has a potential for opening new research directions aimed at understating the role of neural noise in various cognitive processes. The overall aim is to provide a new conceptual framework for linking two infrequently linked brain domains: perception and intelligence.

Skills Required

Stimulus/response programming (MATLAB), working with human subjects

Background paper: Faisal, A. A., Selen, L. P., & Wolpert, D. M. (2008). Noise in the nervous system. Nature Reviews Neuroscience, 9(4), 292-303.

UR Expo Poster (PDF)