I think the most important thing to remember is: it is ALWAYS subjective. If you remove an action from its context, you will always arrive at the wrong answer. The most obvious example is punishment. A punishment is supposed to be bad to the punished. It is a negative that is supposed to be skillfully deployed like a hammer strike. The punished can always claim to be receiving a bad thing, because they are receiving a thing that'd normally be bad.

Punishment itself is not immoral no matter how much any specific person is unjustly punished. When analysing morals, you cannot confuse reciprocity or punishment for origination. Sure, there are plenty of situations where a negative punishment only succeeds in adding negative things to the world, but that's why it's a dangerous tool.

Sooo many people confuse axioms of, "do no harm" with, "I am completely against punishment and the concept of police and prison". It's ... sad, really. So many people want to ignore the nuance, but moral axioms are fundamental pivot points, NOT hard and fast rules.