go hard with

go hard with (someone)

To be difficult, problematic, or unpleasant for someone; to be to someone's disadvantage, harm, or misfortune. It will go hard with John if he doesn't realize that Tommy is only interested in him for his money. It's going to go hard with us if the government cuts our funding any further.
See also: go, hard
Farlex Dictionary of Idioms. © 2015 Farlex, Inc, all rights reserved.

go hard with

Fare ill, be to one's harm or disadvantage. For example, If this case gets to a jury, it will go hard with the defendant. [First half of 1500s]
See also: go, hard
The American Heritage® Dictionary of Idioms by Christine Ammer. Copyright © 2003, 1997 by The Christine Ammer 1992 Trust. Published by Houghton Mifflin Harcourt Publishing Company. All rights reserved.
See also:
References in classic literature ?
"If there is any question about England in all this, and we were recognised, I fear it would go hard with us."
Therefore let us make for the Danaans, that it may go hard with them ere they bear away dead Patroclus to the ships."
"Well, sir, to be quite plain with ye, I am one of those honest gentlemen that were in trouble about the years forty-five and six; and (to be still quite plain with ye) if I got into the hands of any of the red-coated gentry, it's like it would go hard with me.
US media have emphasised that the teams take different approaches to their Wednesday workout, as Seattle go hard with defence and offence working against each other, while the Pats take care not to overwork.
While we'd love to support your claims, Dan, we're always suspicious when locals go hard with the "no deer here" stump speech.
You can also see him at various venues throughout Atlanta and the Southern Region on his Future Icons of the South Tour beginning December 7th with Hard Work hosting Atlanta Urban Wear's Radio station with his self-entitled show: Go Hard with Hard Work and Atlanta Urban Wear's Lyrically Skilled Live Television Broadcast.