Rarity is usually a good thing for collectors, but memorabilia experts say the Confederate flag market is “softening”—a polite way of saying sales are plummeting. And that can’t be pinned on the recent controversy. In 2007, a Confederate flag in moderate condition commanded $77,000 at Heritage Auctions in Dallas. A similar flag brought only $50,000 in 2010. “Had it sold at the same auction in 2007, it would’ve been in the $77,000 price range as well,” says Marsha Dixey, the consignment director for the auction house. Heritage, founded in 1976, boasts over $900 million in revenue a year and employs hundreds of memorabilia experts. It’s also the largest collectibles auctioneer in the world, which is why so much Civil War memorabilia ends up there.

A lot of factors contribute to the changing market, the most obvious being the economic downturn of 2008. However, the generation that typically collected Civil War memorabilia in general—and Confederate flags in particular—is also dying out without being replaced by a younger generation of collectors. “A 40-year-old is not as interested as an older person who remembers their grandfather talking about the war,” Dixey explains.

Collectors say the culture is also changing. “It has a lot to do with political change, especially with what happened recently with Charleston. That will impact flag sales,” explains Michael Collins, executive director of the Civil War Antiques Preservation Society. “That flag should never have been flying on the statehouse. It’s a battle flag, and that sends the message that you are going against the Union.”