To invite another person to one's home. A noun can be used between "ask" and "down." I've been asked down to the MacLeods' farm this weekend. Haven't I asked you down to our new house before?
ask someone down
to invite someone to come to one's home [for a visit]. (Usually said when someone must go to a lower level, travel south, down a hill, or into the country for the visit.) Sam asked us down for Friday evening. Shall we go? We asked down some old friends for the evening.