I'll have to beg off

I'll have to beg off.

Fig. a polite expression used to turn down an informal invitation. Andrew: Thank you for inviting me, but I'll have to beg off. I have a conflict. Henry: I'm sorry to hear that. Maybe some other time. Bill: Do you think you can come to the party? Bob: I'll have to beg off. I have another engagement. Bill: Maybe some other time.
See also: beg, have, off