A4 Refereed article in a conference publication
Building the Penitentiary Document Corpus (PeDoCo) for NLP: Balancing Data Complexity and Uniform Data Structure
Authors: Kupari, Hanna-Mari; Korkiakangas, Timo; Laippala, Veronika
Editors: Mari Väinä
Conference name: Digital Humanities in the Nordic and Baltic Countries Conference (DHNB 2025)
Publisher: University of Oslo Library
Publication year: 2025
Journal: Digital Humanities in the Nordic and Baltic Countries Publications
Book title : Digital Humanities in the Nordic and Baltic Countries Publications 7 (2)
Journal name in source: Digital Humanities in the Nordic and Baltic Countries Publications
Volume: 7
eISSN: 2704-1441
DOI: https://doi.org/10.5617/dhnbpub.12301
Web address : https://doi.org/10.5617/dhnbpub.12301
Self-archived copy’s web address: https://research.utu.fi/converis/portal/detail/Publication/485024197
This paper describes the process of creating a TEI XML corpus of late medieval Latin documents for NLP tasks from books in PDF format. The documents of the Apostolic Penitentiary (a tribunal of the Catholic Church responsible for granting absolutions, dispensations, and indulgences) have been originally published as printed books. For the purposes of this corpus, they were derived from PDF files used for proofreading before printing. These editions, containing 1,511 documents and 211,398 words, are designed by and for human scholars engaged in close reading. As a result, they encode implicit semantic information through typographical features such as page layout and italics, which are sometimes inconsistent. Although human readers, equipped with holistic understanding, can interpret such variations, NLP tools require unambiguous, text-only input.
We report in detail the process of transforming the PDF editions into a structured, machine-readable, and openly accessible corpus. Our approach combines a rule-based workflow using regular expressions with close reading and manual corrections. Such conversion procedures, which are highly time-consuming and require in-depth knowledge of medieval Latin philology and manuscript studies, are regrettably seldom made explicit, despite their vital role in ensuring the reproducibility and scalability of research.
Downloadable publication This is an electronic reprint of the original article. |