Everyone considers themselves libertarian on the issues they are libertarian about. If you think government shouldn’t collect your e-mail and phone logs, you’re libertarian on national-security issues. If you think you have a right to carry a firearm, you’re libertarian about guns. And so it goes with drugs, property rights, free speech, health care, etc. Conservatives are very libertarian about some things and very conservative about others. Ditto liberals and most socialists.
Ideologically consistent libertarians — i.e., people who want freedom across the board — are very vocal, but they are far from legion. And even among the faithful there is still considerable disagreement about issues like abortion or drug legalization beyond marijuana. In principle most Americans simply want government to do good where it can and do no harm anywhere else.
Moreover, people want to maximize freedom in the abstract, but they are loath to pay much of a price for it in their own lives (hence the famous 1964 finding by social psychologists Lloyd Free and Hadley Cantril that Americans are operationally liberal but ideologically conservative).
I wish it were otherwise, but people tend to be libertarian only after it’s demonstrated to them that the government can’t deliver the results they want.