Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings

Learning representations of words in a continuous space is perhaps the most fundamental task in NLP, however words interact in ways much richer than vector dot product similarity can provide. Many relationships between words can be expressed settheoretically, for example adjective-noun compounds (eg. "red cars"⊆"cars") and homographs (eg. "tongue"∩"body" should be similar to "mouth", while "tongue"∩"language" should be similar to "dialect") have natural set-theoretic interpretations. Box embeddings are a novel region-based representation which provide the capability to perform these settheoretic operations. In this work, we provide a fuzzy-set interpretation of box embeddings, and learn box representations of words using a set-theoretic training objective. We demonstrate improved performance on various word similarity tasks, particularly on less common words, and perform a quantitative and qualitative analysis exploring the additional unique expressivity provided by WORD2BOX. * *Equal Contributions. Lorem ipsum dolor sit amet , consectetur ad ip is c in g el i t . Curabitur viverra pretium diam, nec sodales urna semper non. Nunc ut semper arc u . Vestibulum non fringilla lacus. Nullam iaculis leo hendrerit arc u finibus imperdiet. Proin eget quam sit amet mi tempus interdum ac at sapien. Nulla facilisi. Aenean pretium orci a au gu e maximu s males u ad a. Quisque pretium, dui non tempor luctus, magn a mas s a varius sapien, consectetur dictum velit dolor el emen t u m mi. Phasellus vel dui non es t ac c u ms an gravida. Morbi blandit diam eget orci posuere, in eu i s mo d purus finibus. Vestibulum condimentum id ipsum dictum porta. Aenean in lectus nisi. Pellentesque el emen t u m el i t ac risus pretium vehicula. Curabitur ut dapibus mi. Fusce in volutpat felis. Morbi at eu i s mo d el i t . Vestibulum males u ad a quam quis justo porta consectetur. Duis mo llis libero eget sapien aliq u et , at vehicula tellus consectetur. Maecenas id au c t o r nisi, id aliq u am tellus. Quisque lectus el i t , viverra at el emen t u m a, imperdiet a lorem. Vestibulum in justo sit amet mas s a sodales viverra. Aenean eget nibh tincidunt, tempor magn a et , pharetra sapien. Donec tempus nibh iaculis maximu s dapibus. Fusce ultrices tortor nec odio varius, vitae convallis met u s sagittis. Aliquam suscipit sed an t e at gravida. Integer vitae nisl hendrerit, pellentesque nibh non, lobortis justo. Phasellus mo les t ie lectus eu tempus congue. Sed pharetra ullamcorper feugiat. Nunc libero velit, aliq u et a nisi nec, aliq u et pharetra nibh. Pellentesque magn a neque, lobortis eu lectus sit amet , dignissim semper ex. Sus pendis s e faucibus varius ex at eges t as . Duis non vestibulum el i t , eget vehicula nisi. Sed posuere ef f i c i t u r dolor vel dapibus. Nam quis dictum an t e. In pulvinar varius blandit. Pellentesque mo les t ie orci mo llis , pharetra lacus ut, laoreet leo. Mauris non ornare mau ris . Donec el ei f en d dolor quis an t e ullamcorper luctus. Maecenas finibus eu en i m quis ac c u ms an . Interdum et males u ad a fames ac an t e ipsum primis in faucibus. Et iam rhoncus eu tellus non pellentesque. Pellentesque sollicitudin nulla el i t , at gravida es t laoreet a. Sed dignissim in libero a laoreet. Integer vitae purus id leo fringilla mat t is id at lacus. Nunc ac quam ut risus sollicitudin lacinia vitae a velit. Nulla facilisi. Phasellus sit amet tempor ex. Aliquam erat volutpat. Ut consectetur arc u vitae turpis gravida, a placerat dui laoreet. Curabitur maximu s vehicula leo ac commodo. Aliquam erat volutpat. Nulla nec lectus in diam mat t is vehicula. Nulla facilisi. Maecenas tempus scelerisque aliq u am. Vivamus lobortis lobortis viverra. Proin vulputate neque at tempor vehicula. Aliquam placerat urna sit amet porttitor mat t is . Morbi eget mat t is justo. Lorem ipsum dolor sit amet , consectetur ad ip is c in g el i t . Aenean et