When you’re extrapolating so much from 798 people, you have to make a lot of assumptions about what the demographic makeup of an electorate will be and what type of people will be voting. One of the assumptions most pollsters made was that they used a model based on a previous off-year election when Gov. Steve Beshear trounced his weak Republican opponent by more than 20 points. So some pollsters were using a model that assumed an electorate that was more inclined to vote Democratic than it actually was.
It’s difficult to determine what model to use and what assumptions should go into that model. Which is why polling is at least as much art and science, and some pollsters do better gauging certain electorates than others. What very few pollsters figured out in this election was how deeply unpopular President Obama is in Kentucky and how that affected voters’ willingness to support his party, by voting for a Democrat or even by showing up to vote.
Polling is like everything else in life — people are afraid to get too far away from the herd. If you stick to the convention wisdom and get it wrong, you’re in pretty safe company. If you have different assumptions, you’re taking a risk. Again, all this is fine and explains why polling is such a big business in political campaigning.