in fairness

in (all) ˈfairness (to somebody)

used to introduce a statement that defends somebody who has just been criticized, or that explains another statement that may seem unreasonable: In all fairness to him, he did try to stop her leaving.