In the currently booming field of interdisciplinary memory studies, memory hasbeen conceptualised in a plethora of ways. The concept of cultural memorydraws attention to the ways in which memory is not merely a psychological phe-nomenon but always already culturally mediated social phenomenon. WhileMaurice Halbwachs (1925) argued that we remember in social frameworks, morerecent discussions on cultural memory have emphasised that we remember incultural“webs of meaning”(Erll 2008, 4) and that our memory is mediated by“cultural memorial forms”(Laanes and Meretoja 2021). What most socially andculturally oriented approaches to memory agree on is that the individual and thesocial/cultural are entangled in processes of remembering. The sphere of the cul-tural is no longer seen to be limited to cultural artefacts; instead, it is widely rec-ognised that the whole life world in which we live is culturally constituted–thecultural consists of the intersubjective sphere of sense-making practices thatshape our experience and mediate our relationship to the world. Literature is oneimportant forum of cultural memory, but our everyday processes of remember-ing are also culturally mediated and participate in shaping cultural memory; liter-ature, in turn, not only reflects these everyday processes but can also criticallyengage with them and mould them in new directions. In this chapter, I lay out ahermeneutic approach to cultural memory, agency, and historical responsibility. Idelineate the idea of memory as an interpretative activity and discuss literatureas a specific practice of understanding past worlds as spaces of possibility inways that allow us to reflect on how the past implicates us in the present. In thelatter part of the chapter, I develop these ideas in dialogue with Jenny Erpenbeck’s novelHeimsuchung(2008,Visitation), which deals with issues of memory,agency, and responsibility by placing as its protagonist a house by a lake in Bran-denburg, in the former East Germany.