The Obama officials, including holdovers who transitioned into the Trump administration, pulled that off by intimidation: not-so-subtle suggestions that they could disclose damaging allegations at any time (e.g., the notorious “pee tape”), and that White House efforts to inquire into the scope of the investigation would be portrayed as criminal obstruction.
Prior to the 2016 election, the FBI intentionally concealed the existence of the Trump-Russia probe from the congressional “Gang of Eight” (the bipartisan leadership of both houses and their intelligence committees). Senior Republicans were thus kept in the dark regarding purported suspicions that the Republican presidential campaign was a Russian front, unable to pose tough questions about the probe’s gossamer predication.
Crucially, the Trump-Russia fabulists managed to sideline two Trump loyalists who would have been positioned to thwart the effort: national-security adviser Michael Flynn and Attorney General Jeff Sessions. That left in place Obama holdovers and Trump-appointed placeholders. They were indifferent to Trump himself and cowed by the prospect of being framed as complicit in a Trump–Russia conspiracy, or a cover-up.
The paper record is profoundly embarrassing, so it is only natural that the FBI and Justice Department resisted its disclosure. But documents about the investigation were demanded by congressional investigators starting years ago — particularly by the investigation led in the House by then–Intelligence Committee chairman Devin Nunes (R., Calif.).