A4 Vertaisarvioitu artikkeli konferenssijulkaisussa
Pretrained Knowledge Base Embeddings for improved Sentential Relation Extraction
Tekijät: Papaluca Andrea, Krefl Daniel, Suominen Hanna, Lenskiy Artem
Toimittaja: Samuel Louvan, Andrea Madotto, Brielen Madureira
Konferenssin vakiintunut nimi: Annual Meeting of the Association for Computational Linguistics
Julkaisuvuosi: 2022
Kokoomateoksen nimi: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop
Tietokannassa oleva lehden nimi: PROCEEDINGS OF THE 60TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022): STUDENT RESEARCH WORKSHOP
Aloitussivu: 373
Lopetussivu: 382
Sivujen määrä: 10
ISBN: 978-1-955917-23-0
DOI: https://doi.org/10.18653/v1/2022.acl-srw.29
Verkko-osoite: https://aclanthology.org/2022.acl-srw.29/
Rinnakkaistallenteen osoite: https://research.utu.fi/converis/portal/detail/Publication/176250533
In this work we put forward to combine pre-trained knowledge base graph embeddings with transformer based language models to improve performance on the sentential Relation Extraction task in natural language processing. Our proposed model is based on a simple variation of existing models to incorporate off-task pre-trained graph embeddings with an on-task finetuned BERT encoder. We perform a detailed statistical evaluation of the model on standard datasets. We provide evidence that the added graph embeddings improve the performance, making such a simple approach competitive with the state-of-the-art models that perform explicit on-task training of the graph embeddings. Furthermore, we observe for the underlying BERT model an interesting power-law scaling behavior between the variance of the F1 score obtained for a relation class and its support in terms of training examples.
Ladattava julkaisu This is an electronic reprint of the original article. |  |
