← List all talks ...

Design and Analysis of Decentralized Learning Enabled Multi-Agent Systems

Chinmay Maheshwari – PhD Student, UC Berkeley

Thu, 14-Mar-2024 / 4:00pm / Packard 202

(Speaking instead of Lillian Ratliff)

Abstract

The pervasive integration of Machine Learning (ML) and Artificial Intelligence (AI) into societal-scale systems is reshaping our interactions within physical infrastructure and other online services. Unfortunately, the standard ML paradigm, relying on a stationary environment, proves inadequate in such contexts. For instance, various algorithms independently learn and adapt in these dynamic, uncertain, and resource-constrained environments in a decentralized manner, i.e., without communicating and coordinating with one another. Such interactions lead to non-stationarity in the learning data received by any algorithm. Drawing on examples from my research, I will highlight how to incorporate tools from diverse disciplines, such as machine learning, algorithmic game theory, market design, optimization, and dynamical systems, to effectively overcome the challenges associated with design and analysis of decentralized learning algorithms. Specifically, I will focus on two vignettes.

First, I will discuss the problem of decentralized learning in uncertain resource-constrained environments. Particularly, I will focus on learning in two-sided matching markets, where one side of the market, the agents, must learn about their preferences over the other side, the firms, through repeated interaction while competing with other agents for successful matches with preferred firms. I will propose a novel algorithm suitably designed to split up the statistical problem of learning one’s preferences, via noisy observations, from the problem of competing for firms. Additionally, I will discuss the performance of this algorithm in terms of regret guarantees.

Second, I will discuss the problem of decentralized learning in uncertain dynamic environments. I will focus on the long-run outcome of interaction between actor-critic based reinforcement learning algorithms, simultaneous learning and adapting in a decentralized manner under a framework of Markov games. Specifically, I will highlight that such learning algorithms asymptotically converge to the set of Nash equilibria with probability 1.

I will conclude by highlighting some open research opportunities.

Bio

Chinmay Maheshwari is a PhD candidate, advised by Professor Shankar Sastry, in EECS at University of California Berkeley. He obtained M.Tech and B.Tech degrees from Indian Institute of Technology (IIT) Bombay, both in 2019. His research focuses on developing theoretical, algorithmic, and methodological foundations for the design and analysis of learning-enabled multi-agent systems that operate at a societal scale. This involves appropriately accounting for the interplay of algorithms, people, and marketplaces within such systems. On the technical side, his research builds on and extends tools and techniques from machine learning, algorithmic game theory, market design, dynamical systems, control theory and optimization.