It's just that, in my experience, someone saying "don't go out of your way" usually means that they want you to. But, Ianto would probably be up-front with you, right?

Just call me St. Sulu, patron saint of obliviousness. In all honesty, I just don't focus on looks.