In expert–novice research paradigms on the teaching profession, the length of experience is—despite many warnings—used quite often as a key criterion for assigning teachers to the expert group. Using data from the COACTIV project (a study on the professional competence of mathematics teachers in PISA classes) we analyzed the relationship between 13 expertise indicators and length of experience (i.e., years of teaching). By ordering all COACTIV teachers regarding each of the 13 indicators and considering the median rank of a teacher according to these indicators, we were able to form an overall expertise (OE) index. When modeling linear correlations, the 13 indicators as well as the OE index showed negative or zero correlations with length of experience. When allowing for nonlinear models, we identified slight differential relationships between years of teaching and OE within three distinct intervals: a positive relation during the first 8 years of teaching, a negative relation in midcareer, and again a positive relation in the last 10–15 years. Similar curves also emerged in subsamples (academic- vs. vocational-track teachers). We discuss these findings in light of the cross-sectional nature of the COACTIV data. Our results call into question if long experience is a sufficient condition for becoming an expert teacher but strikingly also contradict the widely held view that professional experience is a necessary condition.