SARS-CoV-2 antibody quantity and quality are key markers of humoral immunity. However, there is substantial uncertainty about their durability. We investigated levels and temporal change of SARS-CoV-2 antibody quantity and quality. We analyzed sera (8 binding, 4 avidity assays for spike-(S-)protein and nucleocapsid-(N-)protein; neutralization) from 211 seropositive unvaccinated participants, from the population-based longitudinal TiKoCo study, at three time points within one year after infection with the ancestral SARS-CoV-2 virus. We found a significant decline of neutralization titers and binding antibody levels in most assays (linear mixed regression model, p<0.01). S-specific serum avidity increased markedly over time, in contrast to N-specific. Binding antibody levels were higher in older versus younger participants - a difference that disappeared for the asymptomatic-infected. We found stronger antibody decline in men versus women and lower binding and avidity levels in current versus never-smokers. Our comprehensive longitudinal analyses across 13 antibody assays suggest decreased neutralization-based protection and prolonged affinity maturation within one year after infection.