The two best budget hostels (IMO) in Tel Aviv are Sky Hostel and it's "sister" Momo's hostel, both on Ben Yehuda St, 2 blocks from the ocean. Nearby is also Mike's Place, a grocery store, and even a quick walk can bring you to Allenby St or the Harbor for excellent dance clubs/nightlife (and the "health spas" if you're lonely)
Basically anything you want is close (embassies for visas etc).
http://www.momoshostel.com/
28, Ben-Yehuda St. Tel-Aviv, Israel
http://www.sky1hostel.com/
34, Ben-Yehuda St. Tel-Aviv, Israel
The staff are super helpful and friendly, as are the patrons. They have both dormitory and private rooms (with a lock and a tv). Bathrooms are shared.
If you need luggage locked away, they are very effective at securing it for you. They can also get discount cab rates for the airport (especially if you are leaving on Shabat).
Have fun!
many many weekends spent there.