Of course it does. Maybe you have beautiful cheekbones, or luscious lips, or large eyes, or long lashes, or anything else, or any/all combined.
I've never been told I have a sexy face, and I know I'm generally considered "pretty." The face is what people see and focus on the most when talking with someone. Think about it: if someone said you had sexy boobs or a sexy body, it would be weird if they stared at it while talking to you.
People look at your FACE when they talk to you. So if you have a sexy face...well, props to you, girlfriend!