Humans have limited executive cognitive control or 'willpower' (cf. the psychological literature on ego-depletion). Decision-making and conscious action is draining. It's hard work. The immediate concerns of everyday life can be burdensome enough without adding all the world's ills to one's plate. Again, so long as one is leading a basically decent life, it just doesn't seem reasonable to condemn them or demand that they attend to more pressing concerns elsewhere. Most people have more than enough to attend to already!
So perhaps we should say that one would be 'doing'/bringing about X (or 'allowing' not-X) iff the X option requires more effortful decision-making (i.e. is more ego-depleting) than the not-X option. This could make sense of why bringing about harms is more blameworthy than merely allowing them.
Any thoughts? (Counterexamples or problem cases especially welcome!)