Fine-Grained Distillation for Long Document Retrieval

Long document retrieval aims to fetch queryrelevant documents from a large-scale collection, where knowledge distillation has become de facto to improve a retriever by mimicking a heterogeneous yet powerful cross-encoder. However, in contrast to passages or sentences, retrieval on long documents suffers from the scope hypothesis that a long document may cover multiple topics. This maximizes their structure heterogeneity and poses a granularmismatch issue, leading to an inferior distillation efficacy. In this work, we propose a new learning framework, fine-grained distillation (FGD), for long-document retrievers. While preserving the conventional dense retrieval paradigm, it first produces global-consistent representations crossing different fine granularity and then applies multi-granular aligned distillation merely during training. In experiments, we evaluate our framework on two long-document retrieval benchmarks, which show state-of-the-art performance. * Work done during the internship at Microsoft. † Corresponding author. 1 Each entry of the collection can be any text granularity (e.g., sentence, passage, document) but we take 'document' to denote 'entry of collection' in this paper for clear writing. Document: … The European Renaissance was a time of massive economic and cultural growth following the stagnation of the Middle Ages. Beginning in Italy in the 14th century, the movement spread to all parts of the continent during the next 300 years. The outstanding cultural and artistic heritage of the Renaissance can still be seen today in many of the great cities of the period, including Florence and Venice in Italy, Bruges in Belgium and Toledo in Spain. It All Began in Florence Florence is the city where the Renaissance began, and where it reached its peak in the 15th and 16th centuries under the patronage of the powerful Medici family. Some of the greatest names in Renaissance art are associated with the city, including Leonardo da Vinci, Botticelli and Michelangelo. The poet Dante, the political theorist Machiavelli and the scientist Galileo also lived and worked in Florence. Buildings like the Pitti Palace, Uffizi Gallery and Florence Cathedral are among the masterpieces of Renaissance architecture. The Legacy of Venice To present-day tourists, Venice is renowned for its picturesque canals and its lack of motorized vehicles … Query1: 3 people who were important of the european renaissance time period Query2: what are the italian renaissance cities