Autonomous vehicles (AVs) collect large amounts of raw data from various sensors. Often the data collection, processing and communication raises privacy concerns. These sensor data are integrated with contextual information. This transforms the data into usable metadata that enables real-time processing and analysis. Metadata is privacy sensitive, which requires General Data Protection Regulation (GDPR) compliance and comes with a set of compliance obligations for data controllers and processors. In this research, we propose a conceptual context-based access control framework that enhances the privacy of data collected and processed in autonomous vehicles. The proposed access control framework introduces a layered and modularized architecture. The use of a layered architecture allows the handling of differing privacy risk levels between real-time and historical data. As the potential risk of revealing a specific person in combined data is higher in historical data compared to real-time data, a containment method is proposed for real-time data and sandboxing is proposed for historical data. The data that are processed by AVs consists of personal and non-personal data. While personal data are already a privacy-breaching risk solely, there is also a risk of identifying a specific person by the combination of non-personal data. The proposed access control framework classifies the data, containerizes the data through sandboxing and containment methods, and applies specific access control according to the data context and its privacy implications.