hello.
i’m henry, a final year phd student at the university of edinburgh in natural language processing.
i’m at Cogsci in Rotterdam giving a talk Friday at 11.30am at the LLM session in Mees II – come say hi
we’ve released code for highly-parallelised entropy estimation for pytorch here
how do deep learning models learn to do so much, so well?
My research tries to understand what learning looks like on a representational level. I focus on Information Theory, building efficient, scalable, approaches to interpretability, that let us better understand how large-scale neural networks, work. This helps to provide insight into how learning may work in humans and other species – and helps us build better models by understanding the representational effects of different design decisions.
selected publications
Representations as Language: An Information Theoretic Framework for Interpretability
Representations as LanguageAn Information Theoretic Framework for InterpretabilityThis paper appeared as a talk at the International Meeting of the Cognitive Science Society in 2024.AbstractLarge scale neural models show impressive performance across a wide array of...
Anaphoric Structures Emerge Between Neural Networks
Anaphoric Structures Emerge Between Neural NetworksWithout explicit efficiency pressures.This paper appeared at the International Meeting of the Cognitive Science Society in 2023.AbstractAnaphors are ubiquitous in human language; structures like pronouns and ellipsis...
Compositionality With Variation Reliably Emerges in Neural Networks
Compositionality With Variation Reliably Emerges in Neural NetworksEmergent representations in a multi-agent model are rife with the kinds of variation ubiquitous across natural languages. This paper appeared at ICLR 2024.AbstractWe re-evaluated how to look for...
Meta Learning to Compositionally Generalise
Meta Learning to Compositionally GeneraliseIntroducing domain-general biases via optimisationThis paper appeared as a talk at the Meeting of the Association of Compositional Linguistics (ACL) in 2024.AbstractNatural language is compositional; the meaning of a sentence...