ACL2023

Black-box language model explanation by context length probing

Ondrej Cífka, Antoine Liutkus

2 citations

Abstract

Token-level metrics from the paper <em>Black-box language model explanation by context length probing</em>. The metrics were computed on the UD_English_LinES development set using the <code>preds_to_metrics</code> script from the repository. The archives were created using PyTorch 1.11.0 and can be loaded using <code>torch.load</code>. Each file contains a dictionary mapping metric names to PyTorch tensors. The first two dimensions of each tensor correspond to target token position (within the whole dataset) and context length, respectively. Code for processing the metrics is included in the <code>process_metrics</code> notebook. The metrics are provided for research purposes, in particular to enable reproducing results from the paper without having to recompute or store the model predictions.