There is no such thing as objective morality. One cannot observe that "harmful acts are objectively wrong". The "wrongness" and "rightness" of an action aren't observable, measurable or even well defined properties. It is possible to measure the duration of an action, the energy transformations of the action, the location of an action, ect, but not the morality of an action. What units would you even measure it in? Or is morality a dimensionless property?

From a basic empirical observation of the effects of harm, one can arrive at a moral system based on objective reasoning.

Is this objective moral system utilitarian? Deontological? There is no "objective" argument as to why morality should be either.
How would your objective moral system weigh against incommensurate harms? Maybe its possible to compare the intensities of 2 different physical pains, but how would you compare physical pain with emotional pain? What about weighing pain between different people?

In this way, ideology can be avoided.

The obsession with being "non-ideological" and reducing everything to base science, also known as "positivism" is also an ideology.