Students and faculty at the Naval Postgraduate School (NPS) are working with NASA scientists 60 feet beneath the ocean’s surface at Florida International University’s Aquarius Habitat on a novel effort that seeks to fundamentally transform the possibilities of human-robot collaboration.

Envision a team of Navy divers, or astronauts, arriving at a previously unexplored location. Their objective is to successively map and explore identified areas of interest. But the explorers do not work alone, they work in conjunction with autonomous robots that are capable of “reasoning,” problem solving and assisting their human counterparts.

NPS’ E3 (Exploration in Extreme Environments) and NASA’s Sea Test II programs aim to facilitate this sort of human-robot interaction, ultimately improving diver and astronaut safety and enhancing mapping, navigation and exploration in the undersea and space domains.

“Diver operations are inherently dangerous. Physiological effects limit dive duration and frequency and necessitate a large support crew, thereby increasing operational costs. The sensory-deprived underwater environment also makes navigation, communication and documentation challenging. A robotic diver assistant system can provide autonomous support to dive teams and has the potential to significantly enhance underwater operations,” said NPS Research Associate Professor Noel Du Toit.

“The E3 project is aimed at providing utility to the diver team without burdening the team with vehicle command and control, thereby augmenting the diver team and allowing more effective, efficient and safer operations,” continued Du Toit. “This work seeks to go beyond co-inhabitance of man and machine – our aim is to fundamentally enable the transformative capability of robots.”

NASA Sea Test Program Manager Bill Todd called the collaborative effort, “very rewarding.” He and his team use the Aquarius habitat to confront challenges common to both astronauts and Navy divers.

“Our Sea Test II program is primarily an engineering and astronaut training exercise, our work at the Aquarius habitat very closely mimics the extreme environments of space,” said Todd. “We have a long history of working with the Navy … We are working on the same problems in a similar environment. Working with the Navy is a good fit.

“We can closely replicate the zero gravity or micro-gravity of space in the undersea realm,” continued Todd. “But this is not a simulation, the tasks we do are real and there are real consequences to making mistakes.”

Those consequences, and their mitigation, are a major focus of the collaborative effort. NPS and NASA researchers contend that, given the stakes, every effort must be taken to overcome the limitations that extreme environments pose.

Pursuant to this transformation, researchers at the Aquarius Habitat explored several key unmanned systems areas of research, intelligent autonomy [of unmanned systems], multi-vehicle operations and persistence.

The first step in the E3 field trial involved deploying an NPS Remote Environmental Measuring Units (REMUS) 100 robot to map a large area of the underwater environment. The robot, utilizing NPS-developed algorithms, was tasked with not only mapping the area, but also autonomously identifying obstacles and navigation aids that can be used by divers and other robots.

Next, operators analyze the data gathered by the unmanned system and point out areas that warrant further exploration. The REMUS 100 is redeployed to an area of interest for a closer look and higher resolution data collection. Through subsequent deployments, operators look at increasingly higher-resolution data to further narrow the search area before a Tethered, Hovering Autonomous System (THAUS) is deployed.

“Leveraging this platform’s hovering capability and real-time data streaming, extremely detailed site data can be obtained. All of this exploration occurs before placing a human in this dangerous environment,” said Du Toit.

Finally, a diver is deployed together with the THAUS robot, freeing the human diver to conduct functions suited to human exploration and relegating tasks suited to machines to the diver’s robot assistant. Some of these tasks would be mundane on land, but they present enormous challenges in the undersea or space domains, particularly in cluttered coastal waters and harbors where close proximity operations are necessary.

“Proximal operations with AUVs are a departure from historical underwater robotic applications. Traditionally, the open-ocean model is assumed, a small vehicle operates in a large, uninhabited ocean,” said Du Toit. “The advent of littoral and close proximity dive operations affords new challenges that must be addressed.”

Proximal operations are difficult because they require accurate environmental and positional information to be successful, however the sensory-deprived nature of the underwater environment poses significant challenges – challenges that are exacerbated by the dynamic nature of the maritime domain.

“Techniques for operating close to objects in the presence of changing currents, surges and uncertainty, while ensuring diver and robot safety, are paramount to mission success,” said Du Toit.

To ensure that success, Du Toit is exploring the deployment of both reactive algorithms that can react to unexpected changes in the operating environments and technologies that allow robotic platforms to “reason” about diver’s behavior and identify tasks that will assist them in their endeavors.  

“If we are to truly augment human dive teams, then the robot must be able to interact with its environment, understand its human team member’s intent, and collaboratively perform tasks that augment human operations in a meaningful manner,” said Du Toit.

Back in the Center for Autonomous Vehicles Research lab on the NPS campus, students and faculty evaluate algorithms that support proximal operations with a diver and support robot. While still in its developmental stages, the research hopes to unveil new possibilities on how autonomous underwater vehicles can support Navy diver operations.

But if this level of human-robot collaboration is to be achieved, robots must be able to perform complex tasks such as object recognition, manipulation and vehicle spatial reasoning. In the essentially zero-gravity undersea environment, each time the robot picks up a tool or performs a task, the resulting change in weight and dynamics must be accounted for.

“Adaptive control approaches are achieved by continually re-evaluating the robot’s internal dynamic model. In this way, the robot is actively stabilized even as its performing intervention tasks that would otherwise throw it off balance,” said Du Toit.  

NPS Research Associate Professor Doug Horner insists that the ongoing experiments are breaking new ground because the combined team is not interested in replacing human-controlled operations, but rather augmenting them in a way that enhances human capabilities, arguing, “The human element adds something to these sorts of operations that a machine cannot replicate.

“A human set of eyes brings a contextual perspective and excitement to exploration that is largely absent in robotic exploration,” continued Horner. “Unmanned systems are the basic tools for the next generation of explorers and we want to ensure increase productivity through these human-machine collaborations.”

Central to Horner and Du Toit’s work are the concepts of Simultaneous Localization and Mapping (SLAM). The development of the use of SLAM is an outgrowth of an Office of Naval Research Mine Countermeasure program. NPS’ first development of AUV-based SLAM was in collaboration with the MIT Marine Robotics Lab.

“As the name suggests, SLAM combines together map building and robot localization within the map. SLAM is critically important for environments that don’t have GPS such as in outer space or under the sea,” said Horner. “SLAM is fundamentally no different from age-old techniques in position estimation from reciprocal compass bearings of known objects, but it leverages probabilistic techniques that account for positional uncertainty in the map and the robot’s motion.

“In our case, we use the REMUS with a downward and forward-looking sonar to create a map and navigate according to it,” continued Horner. “In situations where surfacing for a GPS fix is operationally risky, managing position uncertainty, or the robot’s PUC, through the use of SLAM is a frequently necessary option.”

NPS student, Navy Lt. Ryan Hilger, a submariner by trade, is working with Horner on SLAM concepts while pursuing a degree in mechanical engineering. He aims to leverage the ability for multiple Unmanned Underwater Vehicles (UUV) to work together in order to reduce the PUC surrounding submarines.

“As a submariner, I deal with positional uncertainty all the time. The chance to work on something that reduces uncertainty interests me greatly,” said Hilger.

Hilger envisions a host of applications for SLAM-centric, multi-UUV configurations. He is also interested in applying UUV-driven SLAM initiatives to minefield navigation.

“Working together, these UUVs would enter the danger zone and then map and identify the location of each mine,” said Hilger. “The maps that each craft creates would be combined to pin-point the precise location of each mine, after which a stripped down UUV, little more than a torpedo with a guidance system, could lead a swarm of similar systems into the minefield to destroy mines and create a route through which friendly forces could travel.”

Although both NPS and NASA are faced with problems that are specific to the extreme environments in which they operate, researchers from both sides contend that the work at the Aquarius Habitat is an excellent example of what can happen when government agencies combine efforts and work together.

“When we are able to blend agencies, conduct cross-over research, and bring each other’s ideas together it is really effective,” said Todd. “It's the way government should work.”