Whenever I try to post/comment/interact with a community outside my local instance I get a "activitypub_federation::core::activity_queue: Target server (whatever) rejected (My instance/activities/whatever)"

Federation is enabled. I've tried to federate from other communities first, they do indeed connect but I can't post back.