I doubt you're ugly, that's actually a really nice compliment for a stranger to say you have beautiful skin.
Strangers almost never say I am pretty but strangers always tell me that I have beautiful skin all the time. Is it possible that I am ugly but with beautiful skin?
no you are not ugly I am sure you are beautiful and you know what? You are beautiful.
Let me be VERY honest here with you. I have observed the change in people/society for the last 2 decades. Women have made it virtually impossible for a decent man to give a female compliments. If a guy opens his mouth and says you are pretty or opens a door for a woman... They cry sexual harrasment for anything now. Men are afraid to give compliments for fear of being accused of sexually harassing the woman.
Other's opinion of you shouldn't make that much of difference as to shake your confidence? You are what you are no matter what.