Bidirectional Transfer Learning of Multi-Objective Reinforcement Learning for Efficient Online VNF Profiling

Abstract

This research investigates the effectiveness of transfer learning combined with multi-objective reinforcement learning (RL) for profiling diverse VNFs, including Snort (in both Passive and Inline modes) and virtual firewalls. We compare the resource allocation predictions of an RL model with those of a standard machine learning approach, such as a multilayer perceptron (MLP). While MLPs can outperform RL models in certain scenarios, they lack adaptability. Unlike RL, MLPs require retraining when conditions change. To address this limitation, we propose adaptable RL profilers that dynamically allocate resources (CPU, memory, and link capacity) based on the performance needs of the VNFs. The experiments were conducted in four scenarios: two cases of transferring from Snort (Passive Mode and Inline Mode) to a virtual firewall (vFW) and two cases of transferring from vFW to Snort. Our results reveal a trade-off between computational resource utilization (CPU and memory) and link capacity. In the transfer learning scenario from Snort's Inline Mode VNF to vFW, the Q-Learning model with transfer learning (TL) achieved approximately a 20% reduction in vCPU usage compared to the MLP approach. However, it did not perform as effectively as the MLP in reducing link capacity utilization. Conversely, in the transfer learning scenario from vFW to Snort Inline Mode VNF, the Q-Learning with TL model reduced link capacity usage by 20% compared to other models, although it was less efficient in reducing CPU usage.