Evolving and Ensembling Deep CNN Architectures for Image Classification. / Fielding, Ben; Lawrence, Tom; Zhang, Li.

2019 International Joint Conference on Neural Networks (IJCNN). International Joint Conference on Neural Networks (IJCNN) : IEEE, 2019.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Published

Abstract

Deep Convolutional Neural Networks (CNNs) have traditionally been hand-designed owing to the complexity of their construction and the computational requirements of their training. Recently however, there has been an increase in research interest towards automatically designing deep CNNs for specific tasks. Ensembling has been shown to effectively increase the performance of deep CNNs, although usually with a duplication of work and therefore a large increase in computational resources required. In this paper we present a method for automatically designing and ensembling deep CNN models with a central weight repository to avoid work duplication. The models are trained and optimised together using particle swarm optimisation (PSO), with architecture convergence encouraged. At the conclusion of the joint optimisation and training process a base model nomination method is used to determine the best candidates for the ensemble. Two base model nomination methods are proposed, one using the local best particle positions from the PSO process, and one using the contents of the central weight repository. Once the base model pool has been created, the individual models inherit their parameters from the central weight repository and are then finetuned and ensembled in order to create a final system. We evaluate our system on the CIFAR-10 classification dataset and demonstrate improved results over the single global best model suggested by the optimisation process, with a minor increase in resources required by the finetuning process. Our system achieves an error rate of 4.27% on the CIFAR-10 image classification task with only 36 hours of combined optimisation and training on a single NVIDIA GTX 1080Ti GPU.
Original languageEnglish
Title of host publication2019 International Joint Conference on Neural Networks (IJCNN)
Place of PublicationInternational Joint Conference on Neural Networks (IJCNN)
PublisherIEEE
ISBN (Electronic)978-1-7281-1985-4
ISBN (Print)978-1-7281-1986-1
DOIs
Publication statusPublished - 30 Sep 2019
This open access research output is licenced under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

ID: 43383501