Top Republicans in Washington are increasingly skittish that heir apparent Kevin McCarthy lacks the support to succeed Speaker Paul Ryan atop the House Republican conference, paving the way for an ugly leadership fight that yields a dark horse candidate or a strident ideologue.

McCarthy is already working to line up support behind the scenes. But his ascension is unlikely to be smooth, according to interviews with 15 senior Republicans on Capitol Hill and their allies. These Republicans expect a challenger to emerge from the right-wing Freedom Caucus. More intriguingly, they believe that divisions among the GOP’s Establishment bloc may create an opening for a compromise candidate such as Rep. Kevin Brady of Texas, chairman of the powerful Ways and Means Committee, or Rep. Mark Walker of North Carolina, the chairman of the conservative Republican Study Committee.

McCarthy’s office says repeatedly that there is no Leadership race to discuss because Ryan is still in the job. Ryan has backed McCarthy, the House Majority Leader, as his successor in what the Speaker called “a seamless transition.” The White House also seems to support him. McCarthy is close with President Donald Trump, whose daughter, White House senior adviser Ivanka Trump, will join the six-term California congressman at events in Los Angeles and Fresno, Calif., on June 18.