[SEMINAR] Entropy, Compression, Perplexity, and NLP+LLM

Presenter: Prof. Filippi Mignosi

When: November 11, 2025 at 10:00

Where: Aula Seminari, Blocco 0

Estimated duration: about 2 hours (depending on the audience, the duration may vary)

Abstract:
I will discuss the close relationships among the topics listed in the title. In particular, I will show that a standard variant of Perplexity, one of the main metrics used to evaluate the quality of Large Language Models (LLMs), is exactly equivalent to the lossless compression rate achieved by an LLM when it is used as a compressor. We will look together at some relevant challenges such as the Hutter Prize (compression) and BabyLM, and discuss possible implications and future research directions related to language models.