West whipped up a firestorm after he told a Florida town hall on Tuesday night that he believed that about 80 Hill Democrats were members of the Communist Party – which a spokesman later said was reference to the lawmakers who make up Congress’s liberal caucus.

“We hope the people of Florida’s 22nd Congressional District will note that he repeatedly polarizes the American people instead of focusing on their interests,” the caucus’s co-chairs, Reps. Raul Grijalva (D-Ariz.) and Keith Ellison (D-Minn.) said in a statement Wednesday. “When people like Rep. West have no ideals or principles, they rely on personal attacks.”

“We will not engage in base and childish conversations that lower the high level of discourse Americans rightly expect from their representatives,” they added.