Particle Swarm Optimization for Automatically Evolving Convolutional Neural Networks for Image Classification. / Lawrence, Tom; Zhang, Li; Lim, Chee Peng; Phillips, Emma-Jane.

In: IEEE Access, Vol. 9, 18.01.2021, p. 14369-14386.

Research output: Contribution to journalArticlepeer-review

Published

Standard

Particle Swarm Optimization for Automatically Evolving Convolutional Neural Networks for Image Classification. / Lawrence, Tom; Zhang, Li; Lim, Chee Peng; Phillips, Emma-Jane.

In: IEEE Access, Vol. 9, 18.01.2021, p. 14369-14386.

Research output: Contribution to journalArticlepeer-review

Harvard

APA

Vancouver

Author

Lawrence, Tom ; Zhang, Li ; Lim, Chee Peng ; Phillips, Emma-Jane. / Particle Swarm Optimization for Automatically Evolving Convolutional Neural Networks for Image Classification. In: IEEE Access. 2021 ; Vol. 9. pp. 14369-14386.

BibTeX

@article{5982d0fd7a924e3aaa1f4db3abbd1897,
title = "Particle Swarm Optimization for Automatically Evolving Convolutional Neural Networks for Image Classification",
abstract = "Designing Convolutional Neural Networks from scratch is a time-consuming process that requires specialist expertise. While automated architecture generation algorithms have been proposed, the underlying search strategies generally are computationally expensive. The existing methods also do not explore the search space efficiently, and often lead to sub-optimal solutions. In this research, we propose a novel Particle Swarm Optimization (PSO)-based model for deep architecture generation to address the above challenges. Our proposed solution incorporates three new components. Firstly, a group-based encoding strategy is devised, which enforces the candidate networks to always follow the best practices. Specifically, it ensures that the number of groups can be adjusted in accordance with the input image size. By restricting the number of groups, we can adapt the frequency of the pooling operations toward the input image size. As such, it ascertains the position and maximum frequency of the pooling operations always result in a valid network architecture without the need for additional complex governing rules. Secondly, a new velocity updating mechanism is devised, which creates new network architectures by identifying the key network configuration differences. Thirdly, a new position updating mechanism using weighted velocity strengths is devised. Both the velocity and position updating mechanisms facilitate the proposed PSO-based model to search the intermediate positions of the particles{\textquoteright} trajectories, allowing a better trade-off between diversification and intensification to be achieved. We employ eight well-known data sets, including Convex, Rectangles, MNIST and its variants, for model evaluation. The proposed PSO-based model achieves up to 7.58% improvement in accuracy and up to 63% reduction in computational cost, in comparison with those from the current state-of-the-art methods.",
author = "Tom Lawrence and Li Zhang and Lim, {Chee Peng} and Emma-Jane Phillips",
year = "2021",
month = jan,
day = "18",
doi = "10.1109/ACCESS.2021.3052489",
language = "English",
volume = "9",
pages = "14369--14386",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

RIS

TY - JOUR

T1 - Particle Swarm Optimization for Automatically Evolving Convolutional Neural Networks for Image Classification

AU - Lawrence, Tom

AU - Zhang, Li

AU - Lim, Chee Peng

AU - Phillips, Emma-Jane

PY - 2021/1/18

Y1 - 2021/1/18

N2 - Designing Convolutional Neural Networks from scratch is a time-consuming process that requires specialist expertise. While automated architecture generation algorithms have been proposed, the underlying search strategies generally are computationally expensive. The existing methods also do not explore the search space efficiently, and often lead to sub-optimal solutions. In this research, we propose a novel Particle Swarm Optimization (PSO)-based model for deep architecture generation to address the above challenges. Our proposed solution incorporates three new components. Firstly, a group-based encoding strategy is devised, which enforces the candidate networks to always follow the best practices. Specifically, it ensures that the number of groups can be adjusted in accordance with the input image size. By restricting the number of groups, we can adapt the frequency of the pooling operations toward the input image size. As such, it ascertains the position and maximum frequency of the pooling operations always result in a valid network architecture without the need for additional complex governing rules. Secondly, a new velocity updating mechanism is devised, which creates new network architectures by identifying the key network configuration differences. Thirdly, a new position updating mechanism using weighted velocity strengths is devised. Both the velocity and position updating mechanisms facilitate the proposed PSO-based model to search the intermediate positions of the particles’ trajectories, allowing a better trade-off between diversification and intensification to be achieved. We employ eight well-known data sets, including Convex, Rectangles, MNIST and its variants, for model evaluation. The proposed PSO-based model achieves up to 7.58% improvement in accuracy and up to 63% reduction in computational cost, in comparison with those from the current state-of-the-art methods.

AB - Designing Convolutional Neural Networks from scratch is a time-consuming process that requires specialist expertise. While automated architecture generation algorithms have been proposed, the underlying search strategies generally are computationally expensive. The existing methods also do not explore the search space efficiently, and often lead to sub-optimal solutions. In this research, we propose a novel Particle Swarm Optimization (PSO)-based model for deep architecture generation to address the above challenges. Our proposed solution incorporates three new components. Firstly, a group-based encoding strategy is devised, which enforces the candidate networks to always follow the best practices. Specifically, it ensures that the number of groups can be adjusted in accordance with the input image size. By restricting the number of groups, we can adapt the frequency of the pooling operations toward the input image size. As such, it ascertains the position and maximum frequency of the pooling operations always result in a valid network architecture without the need for additional complex governing rules. Secondly, a new velocity updating mechanism is devised, which creates new network architectures by identifying the key network configuration differences. Thirdly, a new position updating mechanism using weighted velocity strengths is devised. Both the velocity and position updating mechanisms facilitate the proposed PSO-based model to search the intermediate positions of the particles’ trajectories, allowing a better trade-off between diversification and intensification to be achieved. We employ eight well-known data sets, including Convex, Rectangles, MNIST and its variants, for model evaluation. The proposed PSO-based model achieves up to 7.58% improvement in accuracy and up to 63% reduction in computational cost, in comparison with those from the current state-of-the-art methods.

U2 - 10.1109/ACCESS.2021.3052489

DO - 10.1109/ACCESS.2021.3052489

M3 - Article

VL - 9

SP - 14369

EP - 14386

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

ER -