Four years after handing Amazon a $600 million contract to develop a cloud-storage service for the US intelligence community that can store information across the full range of data classifications - including Unclassified, Sensitive, Secret, and Top Secret - the NSA announced on Thursday that it has moved most of its mission data to a front-end cloud computing system developed by the agency that's supported by - you guessed it - Amazon and its CEO, Jeff Bezos.
According to NextGov.com, the IC GovCloud, which was created by the NSA but is supported by Amazon's web services, will offer similar hosting services to the other 16 members of the US intelligence community. The advantage of having all of these intelligence agencies using the same system, according to NSA Chief Information Officer Greg Smithberger, is that it will allow analysts from across agencies to share information and "connect the dots" more quickly. But even before the other agencies sign on, the NSA will use the platform to "collect, analyze and store" classified information in a "classified cloud computing environment."
The goal of the platform is to gather all of the signals intelligence that the NSA gathers on foreign targets (and, of course, its myriad spying on the American public) into one centralized location that's easily accessible by its analysts.
The impetus for the multi-year move is getting the NSA’s data, including signals intelligence and other foreign surveillance and intelligence information it ingests from multiple repositories around the globe into a single data lake analysts from the NSA and other IC agencies can run queries against.
"The NSA has been systematically moving almost all its mission into this big data fusion environment," Smithberger told Nextgov in an interview. "Right now, almost all NSA’s mission is being done in [IC GovCloud], and the productivity gains and the speed at which our analysts are able to put together insights and work higher-level problems has been really amazing."
Furthermore, the NSA cloud will employ AI and machine learning techniques to allow analysts to work more quickly than they otherwise would be able to.
Data ingested by NSA has been meta-tagged with bits of information, including where it came from and who is authorized to see it, which ensures analysts only immerse themselves in intelligence they’re cleared to see.
"This environment allows us to run analytic tools and do machine-assisted data fusion and big data analytics, and apply a lot of automation to facilitate and accelerate what humans would like to do, and get the machines to do it for them," Smithberger said. Analysts, he said, can "interactively ask questions" of the data in the cloud environment, and it spits out data in “humanly readable form."
In addition to utilizing some of the commercial technology used by tech firms like Facebook in their data centers, the IC GovCloud will feature proprietary technology developed by the NSA.
The backbone of the system is the same commercial hardware you might see in data centers owned by Facebook, Amazon or other industry titans. But that hardware is blended with NSA-developed custom software, exotic processing, high performance computing and other unique NSA intellectual property.
"It’s really a hybrid of the latest and greatest commercial technology, but a lot of custom NSA technology and a lot of unique development we’ve done to actually create these outcomes," Smithberger said.
Of course, one can't help but wonder what kind of access Jeff Bezos & Co. will have to this new technology. Or, to put it another way, will the most powerful company in America now have access to your girlfriend's nudes?