Definitely go for a used bike and look for better components (shifters, derailer, brakes, etc). A used frame will look less attractive to thieves and be totally fine to ride for a long time. Most bike shop's cheapest bikes will have a new cheapo aluminum frame with crappy components, which will break after a year or so. You do want a good shop to maintain the bike, but you don't have to buy it there.
You don't have to go crazy, but if I were you I'd go down to a bike shop and find out what size fits you and figure out if you want a mountain bike, hybrid, road etc. Once you know that, I'd suggest checking out ebay. They have great search options...I got a very good deal on a used fancy road bike a couple of years ago, like $1300 cheaper than it would've been at a shop.
If you just want a beater to ride around, I'd check out craigslist. The big thing to watch out for is a damaged frame...anything dented or bent is unsalvageable. Also, stand in front of the wheel, spin it and watch where it goes through the brakes to see if the wheel is crazy wobbly. If it's a little off, a bike shop can fix it (it's calling truing the wheel)...or get the person to reduce the price to include buying a new wheel. Rear wheels are much more expensive because you have to replace the gears.
Wear a helmet and happy riding!