Can relate. I personally don't understand the assuming gender thing, because it's sort of a natural response. Now if you're corrected, and you persist, of course that's not okay.

I have had that accidental mixup too and you're right, no harm was meant but some harm was there. It'll definitely stick with you. When I was younger, I had long hair and no facial hair, I've been mistaken for a woman/girl, and you know it's a bit jarring, but you can tell it was an honest mistake.

As long as you're admitting when you're wrong, and not intentionally trying to upset people, you really shouldn't be faulted for much, mistakes happen.