We study narrow ballistic Josephson weak links in a InAs quantum wells contacted by Nb electrodes and find a dramatic magnetic-field suppression of the Andreev reflection amplitude, which occurs even for in-plane field orientation with essentially no magnetic flux through the junction. Our observations demonstrate the presence of a Doppler shift in the energy of the Andreev levels, which results from diamagnetic screening currents in the hybrid Nb/InAs banks. The data for conductance, excess, and critical currents can be consistently explained in terms of the sample geometry and the McMillan energy, characterizing the transparency of the Nb/InAs interface.