If you are this down on yourself that may be a huge reason why others seem to have moved away from you. When you feel this badly on the inside it does show on the outside.
You have got to find a way to learn to love who you are. Instead of thinking badly of yourself you have got to start creating and focusing on all the good things about you. There are good things about you, you've just blinded yourself to them to the point that you don't really like you and others don't really want to be around you.
I'm sure if you asked you would find that alot of people care, they just don't know what they can do to help you. Sometimes when we can't help someone we pull away from them... not the best thing to do, but we don't know how to deal with their negative attitude or depression.
Infants and young children are very smart. They can see things in people much deeper than older people can. There is something good in you... the kids see it, that is why they smile at you. You don't understand because you have blinded yourself to the good in you. They see your face and it makes them happy, it doesn't scare them. They don't see a loser, they see someone who makes them feel happy and want to smile. Focus on that and try to see if you can find in you what they are obviously seeing.
Check into some training or classes to start working with younger kids. See what you can do to get into a field like that. Sure, it doesn't seem like something a guy would do, but who cares.... kids seem to like you and you may be able to do alot of good for them... and in turn for yourself.
In the meantime try to get into some counseling. And if you can't do that then make it a point to start making changes in your attitude. Everyday find something good about yourself (it's there) and look in the mirror and tell yourself over and over that good thing. Find a new one everyday.
When you are out and about take the time to do something nice for someone. You may hate walmart, but it should give you the opportunity to go out of your way everyday to do something nice for someone. As you do nice things and hear the appreciation and thank yous you will soon learn you have some value to this world beyond what you thought.
Best of luck to you and I hope something I said was able to help. I know how you feel.. I've been there. It took alot of time and determination, but today I am the person that is always smiling and happy. If I ain't smiling I have 50 people asking me what's wrong. I am just happier than ever.... you can be too!