But the speaker’s allies have become keenly aware of his precarious political future, the daunting challenge of governing under a President Hillary Clinton or Donald Trump, and the rough task of rebuilding a GOP that many Hill Republicans believe has been overrun by anti-free trade populists who favor bombastic bomb-throwing over thoughtful policy discussions. All while weighing the possibility of his own bid for president, which Ryan did not rule out in a speech on Wednesday.

Whether he goes that route or not, Ryan is likely in for a tumultous next few years. If Trump wins and Ryan retains the speakership, the Wisconsin Republican will be forced to continue to wedge his positions into Trump’s alternate Republican universe.

If Clinton wins, Ryan will have to preside over a slimmed Republican majority, more heavily populated with burn-the-house-down conservatives. He’ll have to cut deals and do business with Hillary Clinton — a woman he’s met with privately just twice — while at the same time keeping conservatives content.

It’s a governing scenario that people close to him are beginning to envision, according to multiple sources in his political orbit — and not a particularly pleasant one.