This month, NOAA’s Joint Polar Satellite System shifted part of its operational data processing ground system to cloud computing. This makes the program the first of NOAA’s National Environmental Satellite, Data, and Information Service (NESDIS) major ground systems to make this transition.
This new system will replace “racks and racks of computers” at the NOAA Satellite Operations Facility, or NSOF, in Suitland, Md. In the previous system, raw data collected from the satellite instruments and received through a system of antennas was then consolidated at NSOF, where it was processed into data products that are used by scientists and assimilated into weather prediction models.
Now, data will instead be processed in the Amazon Web Services GovCloud. The transition has reduced the JPSS footprint of hardware at NSOF and the NOAA Consolidated Back-Up (CBU) facility in West Virginia by 40 percent, said Heather Kilcoyne, JPSS Ground Segment Project Manager.
Each day, the JPSS satellites receive about 400 gigabytes and deliver more than seven terabytes of data to customers.
For NESDIS, this marks the first time the agency has taken a mission operations function at a high security level, and moved it to the cloud, said Irene Parker, Chief Information Officer for NESDIS. “That’s huge for us,” she said.
And the move had to be done without any impact to availability or integrity, Parker said. “It can’t ever go down, and we have to ensure our data stream is accurate, because if it’s not accurate, lives and property can be damaged.” The transition, she added, “has been seamless to the user community. That was one of the key goals.”
This move by JPSS is the first step in a bigger vision for NESDIS, which plans to shift the bulk of its ground systems to cloud computing. That will include data processing, distribution and archive of ground operations—nearly all but the actual commanding of the satellites.
The move to cloud has a number of benefits. Among them, significant long-term cost savings, flexibility to upgrade and faster deployment for science products, Kilcoyne said.
“We want to invest in mission capabilities rather than hardware,” she said. “And if we discover something new and find a better way to create one of our data products, deployment is easier in the cloud. You don’t have to physically install new hardware on the system and make sure it’s running correctly, you just start a parallel set of cloud services, deploy the updated software, and start testing.”
The transition was not without its challenges. The program was in a race to complete all testing, verification, and security measures on a tight two-year deadline, before the existing hardware, which was nearing the end of its lifetime, stopped working. Learning to use new technology and finding the best tools for the system without sacrificing processing speed or security meant a steep learning curve for the engineers on the 120-member team.
“Everybody worked extremely hard on this,” said Marge Ripley, the data product management and services lead for JPSS. “It took lots of long hours to pull it off. But we had a team that was willing to learn in order to pave the way.”
A collaboration between NOAA and NASA, JPSS is the United States’ most advanced series of polar-orbiting environmental satellites. It provides significant technological and scientific advancements for severe weather prediction and environmental monitoring. These data are critical to the timeliness and accuracy of forecasts three to seven days in advance of a severe weather event.