Neural Inference Search for Multiloss Segmentation Models

Sam Slade, Li Zhang, Haoqian Huang, Houshyar Asadi, Chee Peng Lim, Yonghong Yu, Dezong Zhao, Hanhe Lin, Rong Gao

Research output: Contribution to journalArticlepeer-review

21 Downloads (Pure)


Semantic segmentation is vital for many emerging surveillance applications, but current models cannot be relied upon to meet the required tolerance, particularly in complex tasks that involve multiple classes and varied environments. To improve performance, we propose a novel algorithm, Neural Inference Search (NIS), for hyperparameter optimisation pertaining to established deep learning segmentation models in conjunction with a new multi-loss function. It incorporates three novel search behaviours, i.e. Maximised Standard Deviation Velocity Prediction, Local Best Velocity Prediction, and n-dimensional Whirlpool Search. The first two behaviours are exploratory, leveraging Long Short-Term Memory (LSTM)-(Convolutional Neural Network) CNN based velocity predictions, while the third employs n-dimensional matrix rotation for local exploitation. A scheduling mechanism is also introduced in NIS to manage the contributions of these three novel search behaviours in stages. NIS optimises learning and multi-loss parameters simultaneously. Compared with state-of-the-art segmentation methods and those optimised with other well-known search algorithms, NIS-optimised models show significant improvements across multiple performance metrics on five segmentation datasets. NIS also reliably yields better solutions as compared with a variety of search methods for solving numerical benchmark functions.
Original languageEnglish
Pages (from-to)1-15
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Early online date16 Jun 2023
Publication statusE-pub ahead of print - 16 Jun 2023

Cite this