During face-to-face communication, interlocutors’ head and gaze orientation reflect a balance between two competing strategies: one that aims to maximize visual input by directing gaze toward the talker to support degraded acoustic signals through multimodal cues, and another that seeks to optimize acoustic input by orienting the head to improve signal-to-noise ratio. Previous research has shown that this balance is influenced by acoustic challenges such as background noise and hearing impairment, yet most studies have used passive listening tasks and fixed talker positions. This study aimed to enhance ecological validity by analyzing free conversations in a real-life environment, with varying seating arrangements and acoustic conditions.

We collected speech data from 12 triads consisting of one hearing-impaired and two normal-hearing participants who were familiar with one another. Each group engaged in 16 conversations in a canteen, half during noisy lunchtime and half during quieter hours. One participant—either the hearing-impaired individual or an age-matched interlocutor—wore eye-tracking glasses to record gaze and head movement. Seating arrangements were varied such that the eye-tracked participant faced either both interlocutors or one in front and one to the side.

We found that participants gazed less at talkers positioned next to them compared to those seated in front, both during listening and speaking. The offset between head and eye orientation during listening increased with more distal talker positions, higher background noise, and hearing impairment—suggesting a shift toward acoustic optimization. Hearing-impaired participants looked less at their interlocutors while speaking than normal-hearing participants, possibly reflecting increased cognitive load or reliance on other cues. Noise also increased the time spent gazing at the talker during listening, and in noisy conditions, hearing-impaired participants showed more stable gaze behavior with reduced visual scanning.

These findings highlight how acoustic and spatial challenges shape multimodal communication strategies, emphasizing the adaptive coordination of auditory and visual attention in conversational interaction. The reduced visual engagement with non-central talkers and the stabilization of gaze behavior in noise may have important implications for understanding compensatory mechanisms in hearing-impaired individuals and for informing rehabilitation approaches that consider both auditory and visual aspects of communication.