A team of researchers and information technology experts at the Naval Postgraduate School (NPS) has secured nearly $1 million in funding from the High-Performance Computing (HPC) Modernization Program for an ambitious data analytics project dubbed the Data Analysis of Real Time Streams (DARTS).
NPS’ Information Technology and Communications Services (ITACS) partnered with researchers in the computer science and operations research departments on the two-year initiative. To provide the capabilities necessary to support DARTS, improvements needed to be made to the university’s supercomputing cluster, Hamming. During the past quarter, ITACS has co-located the hardware for the new system with Hamming, and have fused its current software code with Apache Hadoop and NPS-developed government software, with testing currently underway.
“Borrowing Hamming’s infrastructure and integrating the supercomputer’s functionalities with the Hadoop system will allow DARTS investigators to conduct big data analytical approaches, real-time processing of streaming data, and offline analysis of historical data. This type of work would have been virtually impossible prior to the DARTS award,” said Dr. Jeff Haferman, Director of NPS HPC and DARTS project lead.
Research efforts, designed to take advantage of the new capabilities that DARTS affords, are slated to begin in the late summer. Researchers are currently set to begin three separate efforts – social media sentiment analysis, bulk data forensics, and image analysis via artificial intelligence – once all testing is complete.
With installation and configuration complete, ITACS’ next phase of preparing the enhanced HPC system is site acceptance, including information assurance, followed by application migration and final testing of the system’s operational capabilities.
“Our greatest challenge right now in testing is how to move data quickly, from ‘point A to point B,’ especially given the sheer volume we will be processing for each research project,” said Haferman. “Twitter alone generates 6 million tweets per minute worldwide.”
Data science and analytics are emerging capabilities that can provide DOD and DON decision makers with a wealth of information. For NPS researchers, DARTS provides faculty on campus with myriad new capabilities ripe for advancement.
“Our shared vision is to create new foundations for computing to make the imaginable unstoppable,” said Joe LoPiccolo, Chief Information Officer and Director of ITACS. “Computing services and devices have already significantly improved our research efforts, but DARTS will propel NPS to the forefront in processing massive data in real-time, and in providing ground-breaking results that will help enable our warfighters today and into the future.”
The DARTS team includes Haferman, Dr. Mathias Kolsch and Michael McCarrin from the Department of Computer Science, and Drs. Samuel Buttrey and Jonathan Alt from the Department of Operations Research.
“Our investigators are maintaining the same standards of excellence in executing their DARTS projects and in producing viable results as in everything they do,” said Haferman. “ITACS HPC staff intend to support them fully in achieving the success they seek.”