Former Gov. Rick Perry, looking to break through in a crowded Republican field for president, is positioning himself as a leading voice against billionaire businessman Donald Trump, whose weeks-old candidacy is already causing headaches for the GOP.

Perry, who has kept a low public profile over the past week or two, delivered a speech Thursday at the National Press Club in Washington, D.C., that was remarkable in its own right. He emphatically urged his party to work harder to connect with black voters and suggested the GOP has occasionally focused too much on state rights at the expense of civil rights.

During the end of his appearance, though, Perry gave reporters another headline in response to a question hinting at Trump’s comments that “rapists” and “killers” are crossing the U.S.-Mexico border. Trump’s remarks, made during his 2016 campaign kickoff last month in New York, have become an uncomfortable topic for Republicans looking to attract more Hispanic voters.

“Let me say,” Perry said, “I don’t think Donald Trump’s remarks reflect the Republican Party.”