This thesis develops a multi-agent extension to the unified framework for sequential decision problems.  We introduce concepts such as interaction layers and belief models, then tie them back into the unified modeling framework for multi-agent problems.  The interaction layers break down the interactions between agents into physical processes and informational processes.  The belief model captures the uncertainty about the state of the world and outlines how to structure a probabilistic mathematical model to capture the complexities associated with the subjective uncertainty of each agent.  The multi-agent extension is structured in such a way to decompose the complex multi-agent models back into the original five components of the unified framework followed by a search over the four classes of policies for each agent.

 

To illustrate the ability of this framework to scale to real-world problems we design two models for epidemics and wildfires.  These problems produce natural multi-agent structure with partial observability, active learning, complex dynamics, and high dimensionality across the state, action, and observation spaces.  The agents in these problems will learn about the environment by collecting valuable information, implementing decisions which directly change the environment, and interacting with each other.  After the formal mathematical modeling, we demonstrate results from python simulations using real data from case studies in each of the application domains.  In both applications, the structure of the problems leads to classes of parameterized direct look ahead approximation policies.