Because it always cones down to an entirely individualized arbitrary standard.

No, it doesn't, and pretending so will not make your case any more convincing.

As neat as the meme is, it just abstracts the problem another level.

A level that still looks very similar to mob rule.

Explain how not accepting hatred and bigotry is "mob rule" and not just "keeping peace", thanks.

The thoughts and ideas that drive progress rarely start from the middle of society.

True but irrelevant. What is your point, here?

And in times of rising authoritarianism the tendency of the center mob to excise and exclude overreaches and leads to an era of fascism.

"In times of rising authoritarianism", sure. But waves of democratic protest for popular movements is not rising authoritarianism. It's rising libertarianism, the exact opposite.

So, rhetorically, what im worried about is how the circle of tolerance always shrinks.

Such as when, and how? When do those pushing for tolerance ever "shrink their circle"? You're just making this up.

Im worried how some use the line of tolerance as a weapon to control thoughts of others.

No, you're not. If you really are, then yes, you are stupid.