Machine learning for resource management in next-generation optical networks

Recent developments in optical networking technology offer the prospect of greater flexibility and configurability over increasingly short timescales in order to address the demands of large capacity, highly bursty, intermittent data transfers, typically in accordance with performance constraints. The aim of this project is to investigate the application of confidence machines to the prediction of highly dynamic traffic behaviour in next generation optical networks and, consequentially, enable these networks to be operated more efficiently. This project is a new collaboration bringing together three strands of recent research: pre-booking resource management, multi-fractal traffic modelling, and confidence machines. As the traffic pattern varies across time granularities, the proposed pre-booking resource management mechanism is hierarchical, whereby the traffic prediction is decoupled into multiple levels. A key point of novelty in the proposal lies in its approach to prediction; namely, the use of confidence information when evaluating plausible alternative resource allocations over a continuum of timescales. Unlike conventional machine learning techniques, the predictions these confidence machines make are hedged: they incorporate an indicator of their own accuracy and reliability. These accuracy reliability measures allow service provider and network carrier to choose appropriate allocation strategies by eliminating unlikely resource demands. Therefore, resource management process can effectively perform a cost-benefit evaluation of alternative actions. The project will employ a "technology-agnostic" approach allowing a number of possible evolution scenarios for next generation optical networking to be considered. The outcome of this research will have important industrial repercussions for optical network efficiency and revenue generation capability, as well as theoretical advances to the evaluation of performance risk in the context of dynamic network behaviour. This latter aspect is likely to have further application, for example, with regard to the performance and resilience of utility computing, and not just the underlying transport.

This research principally focussed on assessing whether predictions of future traffic demand variations can be used to better configure finite network resources to support noticeably higher utilisation levels. Much of the research assumed that the traffic arrival and holding time characteristics would be self-similar in nature. Although this is speculative, it is inline with recent trend data. A variety of prediction models were considered. The most promising one was based on Fractional Auto-Regressive Integrated Moving Average (F-ARIMA). It was able to consistently yield improvements in network utilisation.

Results showed that typically moving less than ~5% of existing connections over the course of several hours can permit a typical network, like NFSNet, with 16 wavelengths per link, to support an additional 10% of connections prior to saturation. However, the magnitude of the benefit could be considered marginal.

Furthermore, the computational complexity of the predictor would likely negate the benefits if the connection arrival density was high, as might be the case in a dynamic ASON environment. Despite this disappointing result, the research provided additional insight into the connection management mechanics and indicated that prediction coupled with limited selective redeployment of existing connections could provide appreciable benefits. This is now the subject of follow-on research.