It is precisely the most offensive speech which needs to be defended, because that is the only speech which ever gets challenged in the first place. If we cave in on this seemingly trivial issue, we have already lost.
And it is often on the most trivial of points that history pivots. Take, for example, the original Free Speech Movement of the mid-1960s, which was the fuse that ignited the social transformations in the second half of that decade. At first, the initial dispute was over something as ridiculous as which student groups were allowed to have a literature table on U.C. Berkeley’s Sproul Plaza, and whether or not the sidewalk bordering the campus counted as university property (where leafletting would be banned) or city property (where it would be allowed). Hardly something worth getting worked up over. But the students pressed the issue, and pressed, and eventually an utterly trivial local dispute became a not-so-trivial local dispute, and when the University caved in, it opened the floodgates to student activism and social upheaval first at Berkeley and eventually across the nation (and world, for that matter).
I posit that this cartoon fiasco may look as trivial now as did the silly Berkeley sidewalk dispute back in 1964, but it could very well morph into a new Free Speech Movement which could affect the course of history just as much as did the first one.