An exclamation of surprise. The phrase is generally thought to have originated as a command to slow down a horse (wherein "Nelly" is the horse's name). They're engaged already? Whoa, Nelly! Whoa, Nelly—what is going on in here?
Rur. Wait! Stop! Tom: When I get that money, I'm gonna get me my own place, and then you and I can get married, and—Jane: Whoa, Nellie! When did I say I was going to marry you? Whoa, Nellie! Did you measure them boards before you started cuttin' 'em?