The human visual system groups local elements into global objects seemingly without effort. Using a contour integration task and EEG source level analyses, we tested the hypothesis that perceptual grouping requires a top-down selection, rather than a passive pooling, of neural information that codes local elements in the visual image. The participants were presented visual displays with or without a hidden contour. Two tasks were performed: a central luminance-change detection task and a peripheral contour detection task. Only in the contour-detection task could we find differential brain activity between contour and non-contour conditions, within a distributed brain network including parietal, lateral occipital and primary visual areas. Contour processing was associated with an inflow of information from lateral occipital into primary visual regions, as revealed from the slope of phase differences between source level oscillations within these areas. The findings suggest that contour integration results from a selection of neural information from lower visual areas, and that this selection is driven by the lateral occipital cortex.