Neural Inference Search for Multiloss Segmentation Models

Semantic segmentation is vital for many emerging surveillance applications, but current models cannot be relied upon to meet the required tolerance, particularly in complex tasks that involve multiple classes and varied environments. To improve performance, we propose a novel algorithm, Neural Inference Search (NIS), for hyperparameter optimisation pertaining to established deep learning segmentation models in conjunction with a new multi-loss function. It incorporates three novel search behaviours, i.e. Maximised Standard Deviation Velocity Prediction, Local Best Velocity Prediction, and n-dimensional Whirlpool Search. The first two behaviours are exploratory, leveraging Long Short-Term Memory (LSTM)-(Convolutional Neural Network) CNN based velocity predictions, while the third employs n-dimensional matrix rotation for local exploitation. A scheduling mechanism is also introduced in NIS to manage the contributions of these three novel search behaviours in stages. NIS optimises learning and multi-loss parameters simultaneously. Compared with state-of-the-art segmentation methods and those optimised with other well-known search algorithms, NIS-optimised models show significant improvements across multiple performance metrics on five segmentation datasets. NIS also reliably yields better solutions as compared with a variety of search methods for solving numerical benchmark functions.