Pretrained Transformers for Text Ranking
BERT and Beyond
Page
%P
-
Chapter and Conference Paper
Towards Automated End-to-End Health Misinformation Free Search with a Large Language Model
In the information age, health misinformation remains a notable challenge to public welfare. Integral to addressing this issue is the development of search systems adept at identifying and filtering out mislea...
-
Chapter and Conference Paper
PyGaggle: A Gaggle of Resources for Open-Domain Question Answering
Text retrieval using dense–sparse hybrids has been gaining popularity because of their effectiveness. Improvements to both sparse and dense models have also been noted, in the context of open-domain question a...
-
Chapter and Conference Paper
Answer Retrieval for Math Questions Using Structural and Dense Retrieval
Answer retrieval for math questions is a challenging task due to the complex and structured nature of mathematical expressions. In this paper, we combine a structure retriever and a domain-adapted ColBERT retr...
-
Chapter and Conference Paper
Pre-processing Matters! Improved Wikipedia Corpora for Open-Domain Question Answering
One of the contributions of the landmark Dense Passage Retriever (DPR) work is the curation of a corpus of passages generated from Wikipedia articles that have been segmented into non-overlap** passages of 1...
-
Chapter and Conference Paper
Squeezing Water from a Stone: A Bag of Tricks for Further Improving Cross-Encoder Effectiveness for Reranking
While much recent work has demonstrated that hard negative mining can be used to train better bi-encoder models, few have considered it in the context of cross-encoders, which are key ingredients in modern re...
-
Chapter
Setting the Stage
This section begins by more formally characterizing the text ranking problem, explicitly enumerating our assumptions about characteristics of the input and output, and more precisely circumscribing the scope o...
-
Chapter
Refining Query and Document Representations
The vocabulary mismatch problem [Furnas et al., 1987]—where searchers and the authors of the texts to be searched use different words to describe the same concepts—was introduced in Section 1.2.2 as a core pro...
-
Chapter
Future Directions and Conclusions
It is quite remarkable that BERT debuted in October 2018, only around three years ago. Taking a step back and reflecting, the field has seen an incredible amount of progress in a short amount of time. As we ha...
-
Chapter and Conference Paper
Another Look at DPR: Reproduction of Training and Replication of Retrieval
Text retrieval using learned dense representations has recently emerged as a promising alternative to “traditional” text retrieval using sparse bag-of-words representations. One foundational work that has garn...
-
Chapter
Introduction
The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in response to a query for a particular task. The most common formulation of text ranking is search, where the search en...
-
Chapter
Multi-Stage Architectures for Reranking
The simplest and most straightforward formulation of text ranking is to convert the task into a text classification problem, and then sort the texts to be ranked based on the probability that each item belongs...
-
Chapter
Learned Dense Representations for Ranking
Arguably, the single biggest benefit brought about by modern deep learning techniques to text ranking is the move away from sparse signals, mostly limited to exact matches, to continuous dense representations ...
-
Book
-
Chapter and Conference Paper
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback: A Reproducibility Study
Pseudo-Relevance Feedback (PRF) utilises the relevance signals from the top-k passages from the first round of retrieval to perform a second round of retrieval aiming to improve search effectiveness. A recent res...
-
Article
Open AccessFrom archive to analysis: accessing web archives at scale through a cloud-based interface
This paper introduces the Archives Unleashed Cloud, a web-based interface for working with web archives at scale. Current access paradigms, largely driven by the scope and scale of web archives, generally invo...
-
Chapter and Conference Paper
Comparing Score Aggregation Approaches for Document Retrieval with Pretrained Transformers
While BERT has been shown to be effective for passage retrieval, its maximum input length limitation poses a challenge when applying the model to document retrieval. In this work, we reproduce three passage sc...
-
Article
Navigation-based candidate expansion and pretrained language models for citation recommendation
Citation recommendation systems for the scientific literature, to help authors find papers that should be cited, have the potential to speed up discoveries and uncover new routes for scientific exploration. We...
-
Article
The ubiquity of large graphs and surprising challenges of graph processing: extended survey
Graph processing is becoming increasingly prevalent across many application domains. In spite of this prevalence, there is little research about how graphs are actually used in practice. We performed an extens...
-
Chapter and Conference Paper
From MAXSCORE to Block-Max Wand: The Story of How Lucene Significantly Improved Query Evaluation Performance
The latest major release of Lucene (version 8) in March 2019 incorporates block-max indexes and exploits the block-max variant of Wand for query evaluation, which are innovations that originated from academia. Th...
-
Chapter and Conference Paper
Which BM25 Do You Mean? A Large-Scale Reproducibility Study of Scoring Variants
When researchers speak of BM25, it is not entirely clear which variant they mean, since many tweaks to Robertson et al.’s original formulation have been proposed. When practitioners speak of BM25, they most li...
