Chris Sims, Assistant Professor, Cognitive Science Department, RPI


Chris Sims, Assistant Professor, Cognitive Science Department, RPI

SAGE 4101

October 24, 2018 12:00 PM - 1:30 PM

Generalization is fundamental to intelligent behavior. For example, if a bird eats a poisonous butterfly, it must learn to avoid preying on that species again, even when no two butterflies look exactly alike. Similarly, in the game of chess a player must generalize intelligently from past experience, as the same board position may never be encountered twice. In this talk I will describe research in both cognitive science and machine learning on a theoretical basis for generalization in reinforcement learning (RL). The approach builds on rate-distortion theory, a branch of information theory that describes optimal communication performance when faced with limits on the ability to store or transmit information. I will describe recent work that has demonstrated that rate-distortion theory provides a novel explanation for the widely known "Universal Law of Generalization" in cognitive science. Building on these results, I will describe early efforts to develop a new family of capacity-limited reinforcement learning algorithms. Compared to standard approaches, capacity-limited RL can achieve faster learning, and better generalization to novel environments. I will conclude by discussing potential avenues for further development of a formal theory of generalization in reinforcement learning.


Download the paper here. 


Add to calendar