“James Rosen is a lot of things, but a criminal co-conspirator he is not,” Graham added. “We’re beginning to criminalize journalism, and I think that should worry us all.”

The Justice Department went after Rosen’s e-mails because of his contact with a State Department official thought to have leaked information about North Korea. Republicans have raised questions about Obama putting Attorney General Eric Holder in charge of that investigation, since it would be looking into his own department.

On the IRS matter, Graham said there was clearly “an organized effort within the IRS” to target Obama’s opponents, and that the White House played a role in developing the culture that allowed that to happen.

Republicans, though, have been wary of calling for a special counsel in that case, at least in part because Holder would be able to appoint that prosecutor.