Determining lifelong physical activity (PA) trajectories and their determinants is essential to promote a physically active lifestyle throughout the life-course. We aimed to identify PA trajectories from childhood to midlife and their determinants in a longitudinal population-based cohort. This study is a part of the Cardiovascular Risk in Young Finns Study. From 1980, a population-based cohort (N = 3596; 1764 boys/1832 girls, age 3-18 years) has been followed up for 31 years. PA indices were formed based on self-reported data (between age 9-49 years) on frequency, duration, and intensity of leisure (during childhood) or high-intensity (at later age) PA and on sports club participation/competitions. PA trajectories were analyzed using group-based trajectory modeling. Childhood (age 12 years), young adulthood (age 24 years), and early midlife (age 37 years) determinants were analyzed. Five PA trajectories were identified: persistently active (6.6%), decreasingly active (13.9%), increasingly active (13.5%), persistently low active (51.4%, reference group), persistently inactive (14.6%). In childhood, rural residential area (OR 0.45, 95% CI 0.21-0.96) and high academic performance (OR 2.18; 95% CI 1.58-3.00) associated with persistently active group. In early midlife, smoking (OR 1.66; 95% CI 1.07-2.58) associated with persistently inactive group, regular alcohol drinking (OR 2.91; 95% CI 1.12-7.55) with persistently active group and having children (OR 2.07; 95% CI 1.27-3.38) with decreasingly active group. High adulthood education associated with both decreasingly (OR 1.87; 95% CI 1.05-3.35) and increasingly (OR 2.09; 95% CI 1.19-3.68) active groups. We identified five PA trajectories from childhood into midlife. Most prominent determinants were academic achievement, education, having children and health habits (i.e. smoking/alcohol use).