Because we live in a world made up of complex sensory information, one of the important challenges of the brain is to filter and synthesize this information in order to guide behavior and shape perception. Stimulation through multiple senses often allows appropriate behavioral responses to be generated under circumstances in which information from one sense is inadequate. Thus, enhanced orientation [1,2], improved target detection [3, 4], and speeded responses [5-14] are among the perceptual and behavioral benefits imparted through effective multisensory integration. This is reinforced by the fact that behavioral responses are typically faster than those predicted by a simple probability summation of responses to unisensory cues alone, indicative of an active integration of these cues [6, 8, 15, 16].
Such behavioral benefits are from underlying neural operations that result from the convergence and integration of inputs from multiple sensory modalities. In order to determine which information from different senses should be perceptually bound as belonging to a common source, the brain relies on certain statistical regularities of sensory stimuli, such as the speed of light vs. sound and the relative neural conduction speeds of each modality. Due to differing propagation speeds, the nervous system allows a certain degree of temporal offset (normally ~300ms) in which multisensory stimuli will be perceptually and neurophysiologically bound [17, 18]. Thus, the timeframe in which multisensory stimuli are highly likely to be integrated and perceived as simultaneous (i.e., belonging to a common source) is known as the “temporal binding window.” This has become a useful construct in assessing an individual’s capacity to determine stimuli that belong together versus stimuli that are disparate. Functionally, the ability to accurately filter environmental stimuli has a profound impact on successfully navigating the world around us.
With its critical role in shaping normal perceptual processes, impairments in effective multisensory integration can generate a sensory environment that is “noisy” and overwhelming, characterized by improper filtering of signal vs. noise and binding of multisensory signals that should not be perceptually bound. Emerging evidence suggests that altered multisensory processing may play a contributory role in numerous clinical conditions including autism spectrum disorder (ASD) [19-21]. ASD is characterized by sensory disturbances [22, 23], and this framework has recently been extended to include changes in multisensory processes and brain networks [19-21, 24-26]. For instance, a previous study demonstrated that for individuals with ASD, the temporal binding window is approximately twice as large compared to typically developed individuals (Figure 1; [21]). Another study has shown that the most significant effect on the TBW in those with ASD is when presented with speech stimuli [27]. In addition, new evidence emphasizes atypical multisensory integration as a contributing factor in the social and linguistic difficulties typically seen in those suffering from ASD [28].
Larger temporal binding windows in individuals with ASD may contribute to the perceptual substrates underlying many pervasive characteristics put forward in the Intense World Theory [29, 30]. This theory proposes that the hypersensitivity and “sensory overload” that autistic individuals experience may be due to neuropathology of hyper-active neuronal circuits. By combining multisensory information that should normally be separated, enlarged temporal binding windows may create an improperly filtered – and therefore more confusing and overwhelming – perceptual world. This in turn may alter the processing of sensory stimuli that would normally be perceived as innocuous (e.g., visits to a store, or social encounters) to become unbearable, aversive, and anxiety-inducing.
Individuals with ASD often show an interest in technology, and social robots were recently discovered to be promising tools in the diagnosis and treatment of ASD [31-35]. Specifically, interactions with robots have been shown to significantly decrease social anxiety in adolescents with ASD (Figure 2; [36]). This age group was of particular clinical interest because adolescents are especially vulnerable to increasingly complex social demands as they transition to adulthood [37]. Failure to smoothly navigate social demands during this developmental transition can be highly detrimental to individuals with ASD, leading to increased social anxiety and depression as well as diminished professional success as adults [38, 39].
Although social robots appear to be effective therapy tools in adolescents, the perceptual mechanism underlying these benefits remains unknown. Given that these social interactions rely on audiovisual communication, we determined to analyze multisensory processing of robot vs. human audiovisual stimuli in order to assess the perceptual benefits that social robots may confer. The underlying hypothesis is that individuals with ASD will exhibit enhanced processing (narrower temporal binding windows) for robotic stimuli compared to human stimuli. Conversely, we hypothesized that human stimuli confer helpful social cues (e.g., body language, emotional prosody of speech) to TD individuals, which would facilitate enhanced processing and result in narrower temporal binding windows.
Social robots are relatively novel but increasingly utilized tools for enhancing social skills and communication for children with ASD. However, to date, measures of their efficacy have been limited to observational assessments. This study applies systems neuroscience tools to target multisensory perceptual substrates that may underlie the therapeutic behavioral benefits of social robots (including increased engagement and attention, decreased social anxiety, and improved human interactions). By identifying, analyzing, and extending these multisensory processing benefits, this research has the potential to strengthen the clinical utility of social robots for individuals with ASD, ultimately facilitating more effective human social encounters, including communication.