DQ Scheuler: deep reinforcement learning based controller synchronization in distributed SDN

Abstract

In distributed software-defined networks (SDN), mul-tiple physical SDN controllers, each managing a networkdomain,are implemented to balance centralized control, scalability andreliability requirements. In such networking paradigm, controllerssynchronize with each other to maintain a logically centralizednetwork view. Despite various proposals of distributed SDNcontroller architectures, most existing works only assume thatsuch logically centralized network viewcanbe achieved withsome synchronization designs, but the question ofhowexactlycontrollers should synchronize with each other to maximizethe benefits of synchronization under the eventual consistencyassumptions is largely overlooked. To this end, we formulatethe controller synchronization problem as aMarkov DecisionProcess (MDP)and apply reinforcement learning techniquescombined with deep neural network to train asmartcontrollersynchronization policy, which we call theDeep-Q (DQ) Scheduler.Evaluation results show that DQ Scheduler outperforms the anti-entropy algorithm implemented in the ONOS controller by up to95.2%for inter-domain routing tasks.