Convolutional and Residual Networks Provably Contain Lottery Tickets

(2022) Convolutional and Residual Networks Provably Contain Lottery Tickets.

Preview

Text ICML22_Burkholz.pdf Download (2MB) | Preview

Abstract

The Lottery Ticket Hypothesis continues to have a profound practical impact on the quest for small scale deep neural networks that solve modern deep learning tasks at competitive performance. These lottery tickets are identified by pruning large randomly initialized neural networks with architectures that are as diverse as their applications. Yet, theoretical insights that attest their existence have been mostly focused on deep fully-connected feed forward networks with ReLU activation functions. We prove that also modern architectures consisting of convolutional and residual layers that can be equipped with almost arbitrary activation functions can contain lottery tickets with high probability.

Item Type:	Conference or Workshop Item (A Paper) (Paper)
Divisions:	Rebekka Burkholz (RB)
Conference:	ICML International Conference on Machine Learning
Depositing User:	Rebekka Burkholz
Date Deposited:	30 Aug 2022 09:48
Last Modified:	30 Aug 2022 09:48
Primary Research Area:	NRA1: Trustworthy Information Processing
URI:	https://publications.cispa.saarland/id/eprint/3755

Actions

Actions (login required)

View Item