According to the nonpartisan Public Interest Declassification Board, a single intelligence agency is producing a petabyte of classified data every 18 months, or the equivalent of 20 million four-drawer file cabinets. The National Archives estimates that, without new technology to accelerate the process, that information would take two million employees a year to review for declassification. Instead, there are just 41 archivists working in College Park, Md., to review records from across the entire federal government — one page at a time.
“Big Data” is a big management challenge. Both Democratic and Republican administrations have chosen to devote more resources to protecting state secrets than to preserving the historical record. The Information Security Oversight Office, the government’s tiny watchdog agency, notes that of the estimated $11.6 billion spent in 2013 to keep information secure, only $99 million was spent on declassification, less than a third as much as 15 years ago. In the late 1990s more than 200 million pages of documents were being declassified each year. Today, that figure has stagnated at around 30 million, despite a huge increase in classified data.
William A. Mayer, executive for research services at the National Archives, said at a recent conference that the current volume of classified information “boggles the mind.” He described the last scene from the Steven Spielberg film “Raiders of the Lost Ark,” where a crate stamped “top secret” was lost among thousands of other crates in a cavernous warehouse. “Spielberg got it right!” Mr. Mayer told the crowd. Our secrets are getting lost, and lost forever.
Join the conversation as a VIP Member