CIRCE at SemEval-2020 Task 1: Ensembling Context-Free and Context-Dependent Word Representations

This paper describes the winning contribution to SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection (Subtask 2) handed in by team UG Student Intern. We present an ensemble model that makes predictions based on context-free and context-dependent word representations. The key findings are that (1) context-free word representations are a powerful and robust baseline, (2) a sentence classification objective can be used to obtain useful context-dependent word representations, and (3) combining context-free and context-dependent representations often improves performance, suggesting that both contain unique relevant information.