Before making offensive or defensive maneuvers, it’s important to run through all the scenarios of what might happen. But when those moves involve massive aircraft carriers and multi-thousand-person crews, it can be a cumbersome, unrealistic task to always rehearse plays and train personnel live.

This is the domain of the The Modeling, Virtual Environments and Simulations (MOVES) Institute, an interdisciplinary academic center at the Naval Postgraduate School (NPS) that works to make it easier for military groups to wargame and practice maneuvers through combined simulation strategies and the integration of artificial intelligence (AI). The institute fosters an open environment for its faculty to explore topics important to the Navy and DOD at large.

“The main thing is to allow people to go off and find problems,” says MOVES Director and NPS Research Associate Professor Dr. Imre Balogh. 

And indeed they do …

Christian Fitzpatrick, a 2009 NPS graduate, is a faculty research associate in the NPS Department of Computer Science who has worked with MOVES since 2018. He was recruited to teach a simulations networking class based on his experience in the Marine Corps Combat Development Command and Office of Naval Research (ONR).

His research combines three types of simulations – live, virtual and constructive, or LVC – together to reimagine training. To explain it in video game terms, think of virtual simulations as the character being controlled by the user and constructive simulations being all the other things in the video game that move and act on their own within the video game world, including the bosses. Fitzpatrick and his team of students are trying to figure out how to integrate live simulations, like running a drill, within constructive and virtual simulations.

“Our LVC research goal at MOVES seeks to enable small units on a live range to be able to act and react in response to the activities of enemy forces modeled in the constructive simulation,” Fitzpatrick explains. “In addition, we want to integrate supporting units through the integration of virtual simulations. The challenge then for us is passing live operator data back into the constructive and virtual simulators. That’s our main research focus.”

The team, made up of Fitzpatrick and his thesis students, use Android devices to track the locations of live players in a virtual, constructive event. The Android devices have software applications that provide geospatial displays that can be passed amongst different players. So, if they place a virtual roadblock or other tactical activity somewhere in the training environment, the players can all get live updates on the location and status of it to aid in their scenario decision-making. They might send virtual missiles or a tactical aircraft ahead of their live trucks to destroy the threats before they get there, for example.

“The use of these simulations allows you to consider training with units that you might never have been able to live,” Fitzpatrick says.

Understanding the ins and outs of LVC is valuable to his students, many of whom will have to conduct their own training as officers.

“Our students are leaving here understanding the whole notion of how LVC works. They get hands-on experience,” Balogh says.

Furthering the dynamic nature of simulations is where Fitzpatrick could see his work merging with colleague Dr. Chris Darken’s work in the future. 

“[Darken] is looking at building entities that can learn over time and be more dynamic,” says Fitzpatrick. “That’s one of the areas we’re looking at potentially integrating these agents into our existing combat simulations so they can be more dynamic and unpredictable.”

Darken, an associate professor in the computer science department at NPS, has been a part of MOVES for 20 years, almost the life of the program, and is one of the core researchers looking into what AI and machine learning (ML) can do for operational planning. Darken is supporting the Naval Surface Warfare Center (NSWC) in Port Hueneme to investigate putting AI into operational planning simulations, and has  been developing the foundation for this effort with the help of several graduate students he has mentored along the way.

Their work, similar to AI programs like Google’s AlphaStar, looks like little blue dots trying to defend themselves against little red dots to the uninformed, but are doing so within different terrains under different objectives. As the simulation runs over the scenario repeatedly, the computer runs different scenarios, eventually learning the best way to complete the mission with the least number of casualties.

There are many different variables that can be added to this simulation such as how many units each party has, and what the terrain looks like; whether there’s a city to defend, and how important defending the city or eliminating adversaries is. The programmer can choose how much of a reward the program will get for completing different activities, thereby creating priorities for the simulation to achieve one task over another. The programmer can also choose to provide negative awards, or basically punish the program for doing certain things. All these will help the program learn to make better decisions as the simulation runs. This approach to ML is called reinforcement learning.

Darken’s work has varied implications for the DOD, according to MOVES leaders.

“There’s like two different possibilities in what his work can do. One implication is, as we do our training, we often want simulated entities and we want our people to be challenged by those simulations. Right now, we have to use real people to play the enemy. If we had AI do some of that, it would make it easier to have people do their trainings,” Balogh explains. “The other possibility is that it is conceivable, given the techniques that he’s using, that people could figure out new tactics that have never been thought about before.”

The first possibility lends itself nicely to improving the constructive simulation side of Fitzpatrick’s work.

Fitzpatrick is also hoping to integrate secure networks into the simulations so military officials with higher security clearance could start training on simulations containing sensitive information.

“A lot of the units we work with are on secure networks,” he says. “That is where they do most of their training … So we’re working to potentially find a way to connect to these secure networks and conduct testing or perform technology insertions into secure live events hosted by the Navy and Marine Corps.”

Fulfilling this part of the project would require additional funding to sustain access to higher clearance material. Fitzpatrick hopes to have a program ready for the next Navy Large Scale Exercise planned for 2024.