President Obama said late Friday that the focus on his administration’s massive Internet and phone surveillance programs have no bearing on his talks with Chinese President Xi Jinping about reining in cyber-spying.

After hours of discussions with Xi at a California desert retreat, Obama called it important to “distinguish between the deep concerns we have as a government around theft of intellectual property or hacking into systems that might disrupt those systems … versus some of the issues that have been raised around NSA programs.”

One of the officials who spoke to me said that because data types are not standardized, the NSA needs several different collection tools, of which PRISM, disclosed today by the Guardian and the Washington Post, is one. PRISM works well because it is able to handle several different types of data streams using different basic encryption methods, the person said. It is a “front end” system, or software, that allows an NSA analyst to search through the data and pull out items of significance, which are then stored in any number of databases. PRISM works with another NSA program to encrypt and remove from the analysts’ screen data that a computer or the analyst deems to be from a U.S. person who is not the subject of the investigation, the person said. A FISA order is required to continue monitoring and analyzing these datasets, although the monitoring can start before an application package is submitted to the Foreign Intelligence Surveillance Court.

From the different types of data, including their credit card purchases, the locations they sign in to the internet from, and even local police arrest logs, the NSA can track people it considers terrorism or espionage suspects in near-real time. An internet geo-location cell is on constant standby to help analysts determine where a subject logs in from. Most of the collection takes place on subjects outside the U.S, but a large chunk of the world’s relevant communication passes through American companies with servers on American soil. So the NSA taps in locally to get at targets globally…

Armed with what amounts to a rubber stamp court order, however, the NSA can collect and store trillions of bytes of electromagnetic detritus shaken off by American citizens. In the government’s eyes, the data is simply moving from one place to another. It does not become, in the government’s eyes, relevant or protected in any way unless and until it is subject to analysis. Analysis requires that second order.

What makes PRISM interesting to us is that it seems to be the ONLY system that the NSA uses to collect/analyze non-telephonic non-analog data stored on American servers but updated and controlled and “owned” by users overseas. It is a domestic collection platform USED for foreign intelligence collection. It is of course hard to view a Facebook account in isolation and not incidentally come into contact with an account that is owned by an American. I assume that a bunch of us have Pakistani Facebook friends. If the NSA is collecting on that account, and I were to initiate a Facebook chat, the NSA would suck up my chat. Supposedly, the PRISM system would flag this as an incidental overcollect and delete it from the analyst’s workspace. Because the internet is a really complicated series of tubes, though, this doesn’t always happen. And so the analyst must sometimes “physically” segregate the U.S. person’s data…

There are many unknowns, of course, and many places where the system could break down. We do not know the minimization rules. They are highly classified. We do not know how long minimized data sits in storage. We don’t know how many NSA analysts are trained to handle U.S. persons’ data, or HOW they are trained. We don’t know the thresholds to determine what the NSA finds to be relevant enough. We don’t know how long the NSA can collect on a target without getting a FISA order, though we do know that they can start collecting without one if the circumstances demand it.

Typically, members of Congress “don’t receive this kind of briefing,” Senate Majority Whip Dick Durbin (D-Ill.) told POLITICO Friday. They wouldn’t have known about the programs unless they were on an intelligence committee, attended special sessions last held in 2011 or specifically asked to be briefed – something they would only know to do if they were clued in by an colleague who was already aware…

Congressional leadership and intelligence committees had access to information about the programs, he said — but the “average member” of Congress likely wouldn’t have been aware of the breadth of the telephone and Internet surveillance…

Like other members who said they learned of the data-gathering efforts when they were revealed in the Guardian and the Washington Post, Schock said the administration classified briefings he’s attended have revealed very little information.

“I can assure you the phone number tracking of non-criminal, non-terrorist suspects was not discussed,” he said. “Most members have stopped going to their classified briefings because they rarely tell us anything we don’t already know in the news. It really has become a charade.”

Faster than you can say evaporation-condensation-precipitation, I expect this week’s exposés to produce additional investigations that will produce more leaks and further scoops about our digital records. This will now fuel new cycles of reporting, leaks and scoops — and another, and another — as new sources are cultivated and reportorial scraps gathering mold in journalists’ notebooks gain new relevance and help break stories.

Greenwald’s storm will continue to rage because, I suspect, the story won’t be limited to just phone records or Web data. Ultimately, it will be about the government’s pursuit of all the digital breadcrumbs we produce as necessary byproducts of day-to-day life — and phone records and Web data are just a small part.

Bank records, credit history, travel records, credit card records, EZPass data, GPS phone data, license-plate reader databases, Social Security and Internal Revenue Service records, facial-recognition databases at the Department of Motor Vehicles and elsewhere, even 7-Eleven surveillance videos comprise information lodes that are of equal or greater value to the national security establishment than phone and Web files.

Thus, the tantalizing prospect of PRISM, and of the whole “finding effort,” is to deny the terrorists the virtual haven that they enjoy throughout the world’s telecommunications spaces — indeed, throughout the whole of the “infosphere,” which includes cyberspace. The piercing of this veil would mark a true turning point in the war on terror, for al Qaeda and other networks simply cannot function with any kind of cohesion, or at any sort of reasonable operational tempo if their communications become insecure. Cells and nodes would be ripped up, operatives killed or captured, and each loss would no doubt yield information that imperiled the network further. Even if al Qaeda resorted to the drastic measure of moving messages, training, and financial information by courier, operations would be so slowed as to cripple the organization. And even couriers can be flagged on “no fly” lists or caught boarding tramp steamers and such.

So for all the furor caused by the PRISM revelations, my simple recommendation is to take a deep breath before crying out in protest. Think first about how the hider/finder dynamic in the war on terror has driven those responsible for our security to bring to bear the big guns of big data on the problem at hand. Think also about whether a willingness to allow some incursions into our privacy might lead to an improved ability to provide for our security, and where that equilibrium point between privacy and security might be. And last, think about the world as it might be without such a sustained effort to find the hidden — to detect, track, and disrupt the terrorists. That would be a world in which they stay on their feet and fighting, and in which they remain secure enough, for long enough, to acquire true weapons of mass destruction. Those of us in the national security business, who know that networks so armed will be far harder to deter than nations ever were, believe that big data approaches like PRISM and its forebears, have been and remain essential elements in the unrelenting and increasingly urgent effort to find the hidden.

What do authoritarian surveillance states do? They act as “information gluttons and information misers.” As gluttons, they take in as much information as possible. More is always better, indiscriminate access is better than targeted responses, and there’s a general presumption that they’ll have access to whatever they want, at any time…

What would a democratic surveillance state look like? Balkin argues that these states would be “information gourmets and information philanthropists.” A democratic surveillance state would limit the data it collects to the bare minimum. Meanwhile, maximum transparency and accountability across branches would be emphasized. Congress and the public would need to be far more involved.

A democratic surveillance state would also place an emphasis on destroying the data that the government collects. Amnesia used to be the first line of defense against surveillance. People just forgot things with time, giving citizens a line of defense against intrusion. In the age of digital technology, however, amnesia no longer exists, so it needs to be mandated by law…

Having a “democratic surveillance” state sounds like an oxymoron, like having a cuddly hand grenade. Perhaps it would be better to just dismantle the surveillance state entirely and be done with it. And indeed removing the laws associated with the Global War on Terror would do much to remove the authoritarian elements of this state.

This is a “Papa knows best” approach to security policy.

We are told that this has helped to keep us safe, and that any loss of civil liberties and sense of privacy is but collateral damage, inconsequential in the grand sweep of things. Many innocents must be violated so that a few guilty people can be stopped. It’s a digital stop-and-frisk, using data trends and a few successes to do huge damage…

This is not a right-left thing. This is a right-wrong thing. This is not about short-term damage to political prospects but about long-term damage to democratic ideals. This is not about any particular person or president or party but about principles and limits.

Obamacare will put into the hands of the IRS medical and health information of an unprecedented level. As bad as leaks as to which websites you visit would be, the threat of leakage of your medical information could be equally devastating to freedom of speech and the political process. It would take a mere nod and a wink to convince someone that participation in the political process was not worth it if the result was the exposure of sensitive medical issues.

You can’t separate the data mining, the culture of intimidation, and criminalization of daily life.

The answer to this problem is not easy, precisely because of the legitimate national security concerns. That where to draw the line may be difficult to ascertain does not mean that a line should not be drawn. The wholesale creation of a national database of everything electronic crosses any reasonable line.

Obama’s response is that we should trust the government.

The Obama scandals tell us otherwise.