The insufficiency of defining goodness as not hurting others is clearly demonstrated by the general assessment of the behavior of most Germans under the Nazis. According to this definition, most Germans during the Holocaust were good people, since they did not hurt anyone. Yet most of us do not consider such Germans good people, clearly indicating that we recognize the impossibility of defining a good person as one who does not hurt anyone.
You do not have to do something bad in order to do bad; you only have to do nothing.