Cognitive and emotional challenges may elicit a physiological stress response that can include arousal of the sympathetic nervous system (fight or flight response) and withdrawal of the parasympathetic nervous system (responsible for recovery and rest). This article reviews studies that have used measures of electrodermal activity (skin conductance) and heart rate variability (HRV) to index sympathetic and parasympathetic activity during auditory tasks. In addition, the authors present results from a new study with normal-hearing listeners examining the effects of speaking rate on changes in skin conductance and high-frequency HRV (HF-HRV). Sentence repetition accuracy for normal and fast speaking rates was measured in noise using signal to noise ratios that were adjusted to approximate 80% accuracy (+3 dB fast rate; 0 dB normal rate) while monitoring skin conductance and HF-HRV activity. A significant increase in skin conductance level (reflecting sympathetic nervous system arousal) and a decrease in HF-HRV (reflecting parasympathetic nervous system withdrawal) were observed with an increase in speaking rate indicating sensitivity of both measures to increased task demand. Changes in psychophysiological reactivity with increased auditory task demand may reflect differences in listening effort, but other person-related factors such as motivation and stress may also play a role. Further research is needed to understand how psychophysiological activity during listening tasks is influenced by the acoustic characteristics of stimuli, task demands, and by the characteristics and emotional responses of the individual.