Going all-grain is the way to save money (on ingredients), and buying bulk (assuming you will use it) helps as well.
Not to mention that once you go AG you never go back.
I buy 55# sacks for $58 - $65, making the per pound cost a dollar plus pennies. The last one was from the health food store that has a homebrew section. Got them to special order it for me and it was cheaper than the lhbs (sorry Ed), though it was on their ordering schedule instead of being there waiting for me.
I got a pound of hops from hopsdirect.com, including shipping, for near $18.
One thing I haven't purchased in bulk is yeast.
You have to commit to a grain and hop combination, but that doesn't have to be a bad thing. Especially if you liven it up by adding varying ounces of crystal, chocolate, black patent, and whatever else strikes your fancy.
My average batch uses 10# grain, 3oz hops, a packet of yeast, and it doesn't have that "tang" associated with extract beers.
All for $15 or so.
Now if you just want to get drunk off cheap swill, and don't care about that nasty extract "tang", I suggest a can of your favorite extract, a 2# bag of brown sugar, boil with your favorite hops, add a packet of yeast, and enjoy.
You will have spent more money than me, and consume a product I would pour down the toilet.
cheers!