What the hell is happening with this Russian mega-hack? Update: A second breach?

“Suspected Russian mega-hack,” I should say. As far as I’m aware, the feds haven’t proved yet that it was Russia’s foreign intelligence service, the SVR. But it has the proverbial “hallmarks” of their work, according to people who understand this stuff.

I’ve stayed away from this story because I lack the technical knowledge to add anything to it and, alas, our government getting hacked by foreign malefactors barely qualifies as “news.” But if you’re avoiding it for similar reasons, the time to tune in is now. The more I read, the more it sounds like the worst security breach in American history. Bar none.

And it’s still going on — maybe. The Russians have done such a masterful job of infiltrating so many systems that American natsec officials aren’t sure yet if they’ve sniffed them out everywhere.

Here’s the story that made me sit up and pay attention:

The Energy Department and National Nuclear Security Administration, which maintains the U.S. nuclear weapons stockpile, have evidence that hackers accessed their networks as part of an extensive espionage operation that has affected at least half a dozen federal agencies, officials directly familiar with the matter said…

They found suspicious activity in networks belonging to the Federal Energy Regulatory Commission (FERC), Sandia and Los Alamos national laboratories in New Mexico and Washington, the Office of Secure Transportation and the Richland Field Office of the DOE. The hackers have been able to do more damage at FERC than the other agencies, the officials said, but did not elaborate…

The attack on DOE is the clearest sign yet that the hackers were able to access the networks belonging to a core part of the U.S. national security enterprise. The hackers are believed to have gained access to the federal agencies’ networks by compromising the software company SolarWinds, which sells IT management products to hundreds of government and private-sector clients.

That last part is the key. Rather than hack different government agencies individually, Russia’s spies targeted the company that provides network-monitoring IT infrastructure to those agencies. Hack SolarWinds and you can use their software as a trojan horse to access the feds’ networks — and many others. It’s not just the federal government that uses SolarWinds, you see. Some state governments do too; at least three have reportedly been breached. Many major companies also use the software. SolarWinds was essentially a skeleton key to unlock secrets both public and private of incalculable security and financial value. Russia stole the key.

And they didn’t do it recently. Apparently the hack happened much earlier this year, with the malware furtively included in the company’s software updates this spring. By simply downloading the update, SolarWinds’s customers unknowingly introduced the code into their systems that gave the hackers access. The piece to read today on this topic is this op-ed by former Trump advisor Tom Bossert, but have a drink beforehand. You’ll wish you had by the end.

The magnitude of this ongoing attack is hard to overstate.

The Russians have had access to a considerable number of important and sensitive networks for six to nine months. The Russian S.V.R. will surely have used its access to further exploit and gain administrative control over the networks it considered priority targets. For those targets, the hackers will have long ago moved past their entry point, covered their tracks and gained what experts call “persistent access,” meaning the ability to infiltrate and control networks in a way that is hard to detect or remove

The actual and perceived control of so many important networks could easily be used to undermine public and consumer trust in data, written communications and services. In the networks that the Russians control, they have the power to destroy or alter data, and impersonate legitimate people. Domestic and geopolitical tensions could escalate quite easily if they use their access for malign influence and misinformation — both hallmarks of Russian behavior.

The Treasury, Homeland Security, State, and Commerce Departments have all been hit; the Pentagon is still checking its systems but it seems safe to assume they’ve been nicked too. As many as 18,000 organizations and 425 Fortune 500 companies may have been infiltrated, according to Bossert. The impression one gets reading his piece is of finding a spot of toxic mold on your ceiling, opening up the wall, and finding the infrastructure so thoroughly infested that you have little choice but to knock down the house and rebuild rather than try to “clean” it. Seriously: Despite the best efforts of Microsoft to seek and destroy the malicious code, cleansing it from infected systems, Bossert seems to think that nothing short of ripping out the infected hardware and replacing it in its entirety will guarantee an end to the infiltration. And that’s even harder than it sounds since some of these government networks are sensitive and will need to remain online until the replacement systems are ready to go. Presumably the Russians, undetected, will go on spying for months. Or years, as that’s how long Bossert thinks it’ll take to fully ascertain which networks were penetrated.

A burning question right now is whether SolarWinds’s software updates were the Russians’ only entry point or whether they’ve since created new, as yet undetected ones into these systems during their many months of unfettered access. Today CISA, the arm of DHS charged with policing for foreign cyberespionage, called the threat from the hack “grave” and acknowledged that other tunnels into the networks may have since been built: “This adversary has demonstrated an ability to exploit software supply chains and shown significant knowledge of Windows networks. It is likely that the adversary has additional initial access vectors and tactics, techniques, and procedures (TTPs) that have not yet been discovered.” We may oust the Russians from our systems and close the gate only to find them back inside six months from now via some exploit we didn’t even know of.

“Doesn’t the government have cyberdefense systems that are supposed to catch this stuff?” you ask. It does, a fancy program called “Einstein.” But Einstein had a flaw: It was designed to detect “new uses of known malware and also … connections to parts of the Internet used in previous hacks,” not novel malware or Internet connections. That’s what the Russians used, it seems. “They used American internet addresses, allowing them to conduct attacks from computers in the very city — or appearing so — in which their victims were based,” the NYT reports. “They created special bits of code intended to avoid detection by American warning systems and timed their intrusions not to raise suspicions — working hours, for example — and used other careful tradecraft to avoid discovery.” A report published two years ago by GAO recommended that Einstein be redesigned to hunt for novel malware too. Didn’t happen. Oops.

The government claims that it hasn’t detected any breach of classified systems, but given how stealthy the Russians were, would they even know? Considering how utterly humiliating this is for the U.S., would the feds even admit that classified databases have been raided? The ominous possibility lurking is that Russia didn’t just make off with mountains of financial and security data. It may have taken proactive measures to alter data in U.S. systems to its own advantage. The greatest cyberspy caper of all time (that we know about) is Stuxnet, the U.S.-Israeli operation to install malicious code in Iranian enrichment facilities so that centrifuges wouldn’t operate properly when enriching uranium. Is there any good reason right now to believe Russia hasn’t done something in that vein to us? How long might they set back some U.S. defense project by covertly changing the specs in a few key places? Could they wreck a civil engineering project to catastrophic effect by doing the same thing?

Again, would we even know if they had? They barely left any digital fingerprints.

“Where is the president of the United States?” asks Jonathan Last today, wondering why the commander-in-chief has had nothing whatsoever to say publicly about this. Not even a generic “we’ll find out who did it and they’ll be punished” threat.

In 45 tweets this week (so far) the president has not said a single word about the record deaths from COVID. Or economic stimulus. Or the Russian hack. Not. One. Word. Instead, it is an endless litany of complaints, self-aggrandizement, and conspiracy theorizing.

We have never seen a dereliction of duty at this scale from an American president. With citizens dying by the thousands every single day and the federal government being raided by the intelligence services belonging to his good friend, he pouts and rages and tweets and tries to overturn a free and fair election in order to break our democratic republic.

There is no precedent for this.

A senator who once famously warned about Russian malevolence to hoots of derision from the other party agrees:

When asked why they love Trump, MAGA voters tend to respond, “He fights!” He fights with the media and the Democrats, yeah. There’s no evidence that he’s fighting with Putin or the Kremlin. No one’s asking him to launch an air raid with a month left in his term, just a forthright statement acknowledging the magnitude of the offense and suggesting what sort of repercussions might be in store for the culprits. But then, this is a guy who once seemed open to forming a joint cybersecurity unit with Russia. Everyone worried at the time that that would give the Kremlin incredible access to U.S. systems but it turns out we needn’t have bothered. They got that access anyway, easily enough.

I’ll leave you with Biden’s statement about this, newly issued this afternoon.

Update: It wasn’t just SolarWinds. They found more than one doorway into U.S. systems.

The Department of Homeland Security said in a bulletin on Thursday the spies had used other techniques besides corrupting updates of network management software by SolarWinds, which is used by hundreds of thousands of companies and government agencies.

“The SolarWinds Orion supply chain compromise is not the only initial infection vector this APT actor leveraged,” said DHS’s Cybersecurity and Infrastructure Security Agency, referring to “advanced persistent threat” adversaries.