Optimizing Distributed Data Processing with Reinforcement Learning

Many of the services we use daily, both online and offline, rely on the processing of massive amounts of data in the background. Given the large scale of the usually distributed (cloud) IT infrastructure underlying this, we aim to minimize the employed computing resources, while maximizing the processing performance. This turns out to be a challenging optimization problem, especially for complex distributed infrastructures and dynamically changing amounts of data to be processed. In this talk, I will explain how we tackle this challenge with reinforcement learning. I will first explain how the infrastructure can be modeled as a graph, connecting different data stores by individual stochastic data processing steps. Based on this, we built a simulation, on which a reinforcement learning agent is trained to dynamically manage the computing resources. I will show first results of the efficiency and performance of the agent compared to a heuristic approach, both based on the simulation as well as tests on real IT infrastructure.