A4 Refereed article in a conference publication

Building the Penitentiary Document Corpus (PeDoCo) for NLP: Balancing Data Complexity and Uniform Data Structure




AuthorsKupari, Hanna-Mari; Korkiakangas, Timo; Laippala, Veronika

EditorsMari Väinä

Conference nameDigital Humanities in the Nordic and Baltic Countries Conference (DHNB 2025)

PublisherUniversity of Oslo Library

Publication year2025

JournalDigital Humanities in the Nordic and Baltic Countries Publications

Book title Digital Humanities in the Nordic and Baltic Countries Publications 7 (2)

Journal name in sourceDigital Humanities in the Nordic and Baltic Countries Publications

Volume7

eISSN2704-1441

DOIhttps://doi.org/10.5617/dhnbpub.12301

Web address https://doi.org/10.5617/dhnbpub.12301

Self-archived copy’s web addresshttps://research.utu.fi/converis/portal/detail/Publication/485024197


Abstract

This paper describes the process of creating a TEI XML corpus of late medieval Latin documents for NLP tasks from books in PDF format. The documents of the Apostolic Penitentiary (a tribunal of the Catholic Church responsible for granting absolutions, dispensations, and indulgences) have been originally published as printed books. For the purposes of this corpus, they were derived from PDF files used for proofreading before printing. These editions, containing 1,511 documents and 211,398 words, are designed by and for human scholars engaged in close reading. As a result, they encode implicit semantic information through typographical features such as page layout and italics, which are sometimes inconsistent. Although human readers, equipped with holistic understanding, can interpret such variations, NLP tools require unambiguous, text-only input.

We report in detail the process of transforming the PDF editions into a structured, machine-readable, and openly accessible corpus. Our approach combines a rule-based workflow using regular expressions with close reading and manual corrections. Such conversion procedures, which are highly time-consuming and require in-depth knowledge of medieval Latin philology and manuscript studies, are regrettably seldom made explicit, despite their vital role in ensuring the reproducibility and scalability of research.


Downloadable publication

This is an electronic reprint of the original article.
This reprint may differ from the original in pagination and typographic detail. Please cite the original version.





Last updated on 2025-10-03 at 11:24