Mean Field Stochastic Control

Abstract

This talk presents the Mean Field (or Nash Certainty Equivalence (NCE)) methodology initiated with Min-Yi Huang and Roland Malhamé for the analysis and control of large population stochastic dynamic systems. Optimal control problems for multi-agent stochastic systems, in particular those with non-classical information patterns and those with intrinsic competitive behavior, are in general intractable. Inspired by mean field approximations in statistical mechanics, we analyse the common situation where the dynamics and rewards of any given agent are influenced by certain averages of the mass multi-agent behavior. The basic result is that classes of such systems possess game theoretic (Nash) equilibria wherein each agent employs feedback control laws depending upon both its local state and the collectively generated mass effect. In the infinite population limit the agents become statistically independent, a phenomenon related to the propagation of chaos in mathematical physics.

Explicit solutions in the linear quadratic Gaussian (LQG) - NCE case generalize classical LQG control to the massive multi-agent situation, while extensions of the Mean Field notion enable one to analyze a range of problems in systems and control. Specifically, generalizations to nonlinear problems may be expressed in terms of controlled McKean-Vlasov Markov processes, while localized (or weighted) mean field extensions, the effect of possible major players and adaptive control generalizations permit applications to microeconomics, biology and communications; furthermore, the standard equations of consensus theory, which are of relevance to flocking behavior in artificial and biological systems, have been shown to be derivable from the basic LQG - NCE equations. In the distinct point process setting, the Mean Field formulation yields call admission control laws which realize competitive equilibria for complex communication networks.

In this talk we shall motivate the Mean Field approach to stochastic control, survey the current results in the area by various research groups and make connections to physics, biology and economics.

This talk presents joint work with Minyi Huang and Roland Malhamé, and Arman Kizilkale, Arthur Lazarte, Zhongjing Ma and Mojtaba Nourian.