Fiche de KARKAR Ahmed skander

Titre : Neural Networks From the Dynamical Transport Viewpoint
Date : 13 octobre 2023
Heure: 14:00
Résumé : The dynamic view of ResNets views them as analogous to the Euler scheme for ODEs. On the other hand, a bias of residual networks towards small perturbations of the input has been observed. In the context of the dynamic view of residual networks, this means a bias towards a small transport cost. In a first paper, we verify experimentally that this bias is beneficial and should be encouraged and we show that forcing the network to approximate an optimal transport map by regularizing its transport cost improves its generalization ability. In a second paper, we show that applying this transport regularization to successive neural modules that don't back-propagate to each other improves the performance of module-wise training, which consumes a lot less memory than end-to-end training. In a third paper, , we propose a detector of adversarial attacks that is based on the view of ResNets as discrete dynamical systems and show that transport regularization makes adversarial detection easier.
Lieu : Sorbonne Université