Audiovisual angle and voice incongruence do not affect audiovisual verbal short-term memory in virtual reality

Virtual reality (VR) environments are frequently used in auditory and cognitive research to imitate real-life scenarios, presumably enhancing state-of-the-art approaches with traditional computer screens. However, the effects of different display technologies on audiovisual processing remain underexplored. This study investigated how VR displayed with an head-mounted display (HMD) affects serial recall performance compared to traditional computer monitors, focusing on their effects on audiovisual processing in cognitive tasks. For that matter, we conducted two experiments with both an HMD and a computer monitor as display devices and two types of audiovisual incongruences: angle (Exp. 1) and voice (Exp. 2) incongruence. To quantify cognitive performance an audiovisual verbal serial recall (avVSR) task was developed where an embodied conversational agent (ECA) was animated to speak the target digit sequence. Even though subjective evaluations showed a higher sense of presence in the HMD condition, we found no effect of the display device on the proportion of correctly recalled digits. For the extreme conditions of angle incongruence in the computer monitor presentation the proportion of correctly recalled digits increased marginally, presumably due to raised attention, but the effect is likely too small to be meaningful. Response times were not affected by incongruences in either display device across both experiments. These findings suggest that the avVSR task is robust against angular and voice audiovisual incongruences, irrespective of the display device, at least for the conditions studied here. Hence, the study introduces the avVSR task in VR and contributes to the understanding of audiovisual integration.

Real-time auralization for performers on virtual stages

This article presents an interactive system for stage acoustics experimentation including considerations for hearing one's own and others' instruments. The quality of real-time auralization systems for psychophysical experiments on music performance depends on the system's calibration and latency, among other factors (e.g. visuals, simulation methods, haptics, etc). The presented system focuses on the acoustic considerations for laboratory implementations. The calibration is implemented as a set of filters accounting for the microphone-instrument distances and the directivity factors, as well as the transducers' frequency responses. Moreover, sources of errors are characterized using both state-of-the-art information and derivations from the mathematical definition of the calibration filter. In order to compensate for hardware latency without cropping parts of the simulated impulse responses, the virtual direct sound of musicians hearing themselves is skipped from the simulation and addressed by letting the actual direct sound reach the listener through open headphones. The required latency compensation of the interactive part (i.e. hearing others) meets the minimum distance requirement between musicians, which is 2 m for the implemented system. Finally, a proof of concept is provided that includes objective and subjective experiments, which give support to the feasibility of the proposed setup.

Autophonic Loudness of Singers in Simulated Room Acoustic Environments

This paper aims to study the effect of room acoustics and phonemes on the perception of loudness of one's own voice (autophonic loudness) for a group of trained singers. For a set of five phonemes, 20 singers vocalized over several autophonic loudness ratios, while maintaining pitch constancy over extreme voice levels, within five simulated rooms. There were statistically significant differences in the slope of the autophonic loudness function (logarithm of autophonic loudness as a function of voice sound pressure level) for the five phonemes, with slopes ranging from 1.3 (/a:/) to 2.0 (/z:/). There was no significant variation in the autophonic loudness function slopes with variations in room acoustics. The autophonic room response, which represents a systematic decrease in voice levels with increasing levels of room reflections, was also studied, with some evidence found in support. Overall, the average slope of the autophonic room response for the three corner vowels (/a:/, /i:/, and /u:/) was -1.4 for medium autophonic loudness. The findings relating to the slope of the autophonic loudness function are in agreement with the findings of previous studies where the sensorimotor mechanisms in regulating voice were shown to be more important in the perception of autophonic loudness than hearing of room acoustics. However, the role of room acoustics, in terms of the autophonic room response, is shown to be more complicated, requiring further inquiry. Overall, it is shown that autophonic loudness grows at more than twice the rate of loudness growth for sounds created outside the human body.

Two simultaneous talkers distract more than one in simulated multi-talker environments, regardless of overall sound levels typical of open-plan offices

The irrelevant speech effect (ISE) characterizes detriment to cognitive task performance in the presence of irrelevant speech. This paper examines whether the ISE varies due to the number of simultaneously active nearby talkers (for up to two talkers), or the overall sound level, within the context of a simulated open-plan office. Two experiments were conducted within a climate-controlled chamber that was set-up as a medium-sized open-plan office. The cognitive tasks performed by the participants included the digit recall task, and a writing task, within a room acoustic simulation of realistic multi-talker speech from spatially separated talkers. Within Experiment 1 (n=60), an increase in the number of talkers from none (T0) to one (T1), and from one to two (T2) simultaneous talkers resulted in statistically significant decline in the digit recall task performances, with effect sizes of 24% (i.e., T1 vs. T0), and 12% (i.e., T2 vs. T1), respectively. The pauses between words during the writing task were similar for T0 and T1, but showed a statistically significant increase within T2 vs. T1, with an effect size of 12%. The findings of Experiment 1 are inconsistent with the maximally distracting status attributed to T1 in some studies, but is consistent with findings in other studies. Within Experiment 2 (n = 62), the cognitive performance in T2 remained largely invariant between 45 and 57 dB (A-weighted sound pressure levels), which represents a typical range of levels within open-plan offices. In general, these findings have relevance for characterizing auditory distraction within complex multi-talker environments; both in laboratory studies and actual open-plan offices. (Abridged version; please see the paper for the full abstract)

Reliability and repeatability of ISO 3382-3 metrics based on repeated acoustic measurements in open-plan offices

This paper investigates variability in the key ISO 3382-3:2012 metrics, based primarily on the repeatability and reliability of these metrics, using repeated measurements in open-plan offices. Two types of repeated measurements were performed in offices, Type1 (n=36), where the same path over workstations was measured from opposite ends, and Type2 (n=7), where two different measurement paths were measured. Overall, most of the Type1 results seem reasonable considering repeats were conducted in complicated room acoustic environments, while Type2 repeats would benefit from larger sample sizes in future studies. Some recommendations are outlined for the ISO 3382-3 methodology vis-a-vis Type1 and Type2 repeats, including future research directions that go beyond increased sample sizes. (This is an abridged version of the abstract. Please see the paper for the full abstract)

Cognitive performance in open-plan office acoustic simulations: Effects of room acoustics and semantics but not spatial separation of sound sources

The irrelevant sound effect (ISE) characterizes short-term memory performance impairment during irrelevant sounds relative to quiet. Irrelevant sound presentation in most ISE studies has been rather limited to represent complex scenarios including open-plan offices (OPOs) and not many studies have considered serial recall of heard information. This paper investigates ISE using an auditory-verbal serial recall task, wherein performance was evaluated for relevant factors for simulating OPO acoustics: the irrelevant sounds including speech semanticity, reproduction methods over headphones, and room acoustics. Results (Experiments 1 and 2) show that ISE was exhibited in most conditions with anechoic (irrelevant) nonspeech sounds with/without speech, but the effect was substantially higher with meaningful speech compared to foreign speech, suggesting a semantic effect. Performance differences in conditions with diotic and binaural reproductions were not statistically robust, suggesting limited role of spatial separation of sources. In Experiment 3, statistically robust ISE were exhibited for binaural room acoustic conditions with mid-frequency reverberation times, T30 (s) = 0.4, 0.8, 1.1, suggesting cognitive impairment regardless of sound absorption representative of OPOs. Performance differences in T30 = 0.4 s relative to T30 = 0.8 and 1.1 s conditions were statistically robust, but not between the latter two conditions. These results suggest that certain findings from ISE studies with idiosyncratic acoustics may not translate well to complex OPO acoustic environments.

Sound in occupied open-plan offices: Objective metrics with a review of historical perspectives

Open-plan offices (OPOs) have been around for more than half a century now, chronicling the vicissitudes of workplace topography amongst other factors. This paper addresses one such factor - the sound environment in occupied OPOs in relation to several objective workplace parameters, using measurements in contemporary OPOs and comparisons with studies over the last 50 years. Omnidirectional and binaural sound measurements were conducted in 43 offices during typical working hours. The results describe variation in several acoustic and psychoacoustic metrics, and present statistical models that predict these metrics as a function of the number of workstations in offices. LA,eq of 53.6 dB is typical for occupied OPOs, with spectral slope of approximately -4 dB/octave. LA,eq values do not vary much over the workplace parameters studied (e.g., floor plate area, work activity, etc), except for -2.7 dB and -4.1 dB differences between offices with/without carpeting, and offices with ceiling absorption but with/without carpeting, respectively; most likely from reduced floor impact noise leading to speech level reduction. Sound fluctuation, as characterised by the metric Noise Climate (NCl: LA10 - LA90) and the psychoacoustic Fluctuation Strength (FS), decreases significantly with increasing number of workstations in OPOs. This suggests lesser auditory distraction in larger offices, which needs further investigation. In terms of historical trends, OPOs have become quieter over the years, especially background noise quantified as LA90, although there are several subtleties. Overall, current findings can inform several OPO design perspectives including policy documents, provide values for laboratory simulations of OPO acoustic environments, help interpret subjective impressions of OPO occupants, etc.