The problem is that Americans use the state as a moral compass. For libertarians, it is often frustrating to explain that advocating for the decriminalization of X is not synonymous with endorsing X. It’s often easier to rationalize away the consequences of enhanced choice than to admit it exists.
Marijuana is, for the most part, is an innocuous habit. But there can be detrimental psychological and physiological effects on the human body after prolonged use. It hinders the mental capacity of people who use it excessively. No doubt, you’ve met some test subjects. Many pro-pot legalization advocates want Americans to believe that nurses, accountants, shopkeeps and local haberdashers make up the majority share of those smoking Carmelicious on weekends. Anyone who’s done any reporting on the issues understands this is preposterous. There are hordes of stoners making a “lifestyle” choice and wasting away. Is it a huge deal? Probably not. Should we criminalize slacking? No. Is it something that should discouraged? Probably. One sort of life you choose might be better than another sort of life. One imagines that most libertarian pundits who argue for legalization have higher degrees to prove it.