Over the past few years, the rapid development of organic-metal halide-based perovskite solar cells (PSCs) has grown ample attention among researchers and proved their great potentialities to predominate the market of the next-generation photovoltaics. However, in comparison with the other structures, the regular planar structured (n-i-p configuration) PSCs have shown the best photovoltaic performances ever (over 25% PCE) though they owe some issues like high cost of synthesis, considerable hysteresis, and performance instability. To overcome such issues, the adoption of the inverted PSCs (p-i-n configuration) has been becoming much more popular day by day. Moreover, rapid improvement in PCE of such devices by using various environment-friendly, cost-effective advanced materials in the individual device layers have made them more contrasting in recent years. This article reviews the recent developments of inverted perovskite solar cells (IPSCs) towards better photovoltaic performance and sustainability especially focusing on device engineering using various advanced and promising materials into the device layers such as perovskite absorber, ETLs, HTLs, and electrodes. Besides, the challenges and recommendations on future research work for the sustainable development of high-performance IPSCs are also discussed in detail.