Virtual Reality (VR) enables the presentation of realistic audio-visual environments by combining head-tracked binaural auralizations with visual scenes. Whether these auralizations improve social presence in VR and enable sound source localization comparable to that of real sound sources is yet unclear. Therefore, we implemented two sound source localization paradigms (speech stimuli) in a virtual seminar room. First, we measured localization continuously using a placement task. Second, we measured gaze as a naturalistic behavior. Forty-nine participants compared three auralizations based on measured binaural room impulse responses (BRIRs), simulated BRIRs, both with generic and individual head-related impulse responses (HRIRs), with loudspeakers and an anchor (gaming audio engine). In both paradigms, no differences were found between binaural rendering and loudspeaker trials concerning ratings of social presence and subjective realism. However, sound source localization accuracy of binaurally rendered sound sources was inferior to loudspeakers. Binaural auralizations based on generic simulations were equivalent to renderings based on individualized simulations in terms of localization accuracy but inferior in terms of social presence. Since social presence and subjective realism are strongly correlated, the implementation of plausible binaural auralizations is suggested for VR settings where high levels of (social) presence are relevant (e.g. multiuser interaction, VR exposure therapy).