Even if the messages are end to end encrypted I wouldn't trust a third party with that data (unless it was a company that does it for a living.)
I'd probably recommend running your own instance, I imagine for that few users it would be pretty cheap. Though maintenance is probably the biggest issue there.
Or just use a third party Matrix server but send client info over email using GPG keys. That would cost you nothing.
I'm seeing this too, it's hard to tell if it's a Sync issue or defederated instances (Beehaw users?)