The intracellular cilia assembly pathway is a complex, multistep process that requires the continuous and coordinated incorporation of membrane material. However, how membrane remodeling occurs during early ciliogenesis is not yet understood. Moreover, the identity of the organelle(s) that supply membrane material for the nascent cilium has yet to be determined. Here, we extend the current model of primary cilia formation by showing that randomly attached distal appendage vesicles and tubules fuse laterally to generate a doughnut-shaped membrane structure. Centripetal fusion events follow to close the central hole. Our data demonstrate that both the Golgi apparatus and endocytotic pathways independently contribute to ciliogenesis. We identify the endocytotic protein GRAF1 as being essential during the early stages of ciliogenesis and for the delivery of plasma membrane-derived material to the developing ciliary membrane. Our three-dimensional ultrastructural analysis uncovers previously unrecognized intermediate stages in the intracellular cilia assembly pathway with GRAF1 as a regulator of ciliogenesis.