Immigrants’ life circumstances both before and after migration are likely to vary considerably by type of residence permit, which may affect fertility behaviour. Yet, the relationship between permit type and fertility is surprisingly underexplored. This study uses Swedish population register data to examine fertility differences by permit type among female immigrants to Sweden. The study also examines variation in fertility patterns within the family migrant category by characteristics of the male partner. The analysis compares how the mean number of children of different groups develops over time, both before and after immigration. The relationship between permit type and fertility is examined separately by geographical origin, to facilitate the disentanglement of permit type and origin effects on immigrant fertility. Results show that the mean number of children is higher among refugees and family migrants than among labour and student migrants. Family migrants experience an increase in fertility tempo shortly after migration, whereas this pattern is less pronounced and often somewhat delayed among labour and student migrants and largely absent among refugees. Among family migrants, women who migrated together with or shortly after their male partner have the highest fertility, whereas family migrants with a Swedish-born partner have lower fertility than other family migrants. This study contributes new knowledge to the understanding of how permit type relates to fertility, by studying fertility differences between more permit categories than earlier studies, by considering time both before and after migration, and by exploring heterogeneity within the family migrant category.