An Unsupervised Query Rewriting Approach Using N-gram Co-occurrence Statistics to Find Similar Phrases in Large Text Corpora




Hans Moen, Laura-Maria Peltonen, Henry Suhonen, Hanna-Maria Matinolli, Riitta Mieronkoski, Kirsi Telen, Kirsi Terho, Tapio Salakoski, Sanna Salanterä

Mareike Hartmann, Barbara Plank

Nordic Conference on Computational Linguistics

2019

Linköping Electronic Conference Proceedings

Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa)

NEALT Proceedings Series

42

131

139

978-91-7929-995-8

1650-3686

https://www.aclweb.org/anthology/W19-6114/

https://research.utu.fi/converis/portal/detail/Publication/44203057



We present our work towards developing a system that should find, in a large text corpus, contiguous phrases expressing similar meaning as a query phrase of arbitrary length. Depending on the use case, this task can be seen as a form of (phraselevel) query rewriting. The suggested approach works in a generative manner, is unsupervised and uses a combination of a semantic word n-gram model, a statistical language model and a document search engine. A central component is a distributional semantic model containing word n-grams vectors (or embeddings) which models semantic similarities between ngrams of different order. As data we use a large corpus of PubMed abstracts. The presented experiment is based on manual evaluation of extracted phrases for arbitrary queries provided by a group of evaluators. The results indicate that the proposed approach is promising and that the use of distributional semantic models trained with uni-, bi-and trigrams seems to work better than a more traditional unigram model.


Last updated on 2024-26-11 at 21:36