Depends on the girl, really. Personally, I've never really cared about that. I've been interested in guys who were kind of the preppy little pretty-boy type, and I've been into guys who think it's perfectly acceptable to throw on a wrinkled T-shirt and jeans that they just picked up off of their bedroom floor. Really, though, I would say that it depends on the girl and what she feels like looks most attractive on a particular guy.
Just my opinion.