albatross around neck
albatross around/round your neck(literary)
something that you have done or are connected with that keeps causing you problems and stops you from being successful
Usage notes: An albatross is a large white bird. In the poem The Rime of the Ancient Mariner, by Samuel Taylor Coleridge, a man on a ship kills an albatross which is then hung round his neck to show that he has brought bad luck.The company that he founded in 1983 is now an albatross around his neck, making losses of several hundreds of thousands a year.