President George W. Bush had plenty of groups he did not meet with or spurned, often changing his mind only after negative public reaction built up. Remember him declining year after year the chance to speak to the National Association for the Advancement of Colored People — an organization that hosted every president since Herbert Hoover? Only in 2006 did he reverse course.
Activist Cindy Sheehan became famous for following Bush to his ranch, asking for a meeting and a hearing. Bush also declined to meet with a bipartisan committee working to rewrite the education bill and, despite repeated requests, refused all but once to meet with the Congressional Black Caucus. Bush even declined to meet with Nelson Mandela in 2003 because the former South African political prisoner opposed the Iraq war.
As House speaker, Newt Gingrich would not meet with the National Endowment for the Arts, calling it a group based on patronage and elitism.
So when it comes to declining meetings with groups you disagree with, Bush and Gingrich set the standard — not this White House.