[rede.APPIA] DaSSWeb – Data Science and Statistics Webinar – 17 June – José Carlos Príncipe – Dynamics of Information in Deep Learning

Title

Dynamics of Information in Deep Learning

Abstract

This talk presents an overview of how information theoretic (IT) concepts and algorithms can be applied to explain the mappings learned in machine learning. The first step is to select an approach to estimate directly from data entropy and mutual information because in machine learning the pdf of the data is normally unknown. Here we will show how Renyi’s entropy and mutual information can be estimated from the eigenspectrum of the Gram matrix of kernel learning. Here we will briefly explain how IT can be used to analyze the dynamics of learning and set up proper topologies in deep learning