Start with the basics: If you are a minimalist, you can get by with less, please make sure you have a helmet.

BIKE - old, new, yelo, red, pink, blue, it is a simple machine with few moving parts. Somewhere between free, $10, and $100 will get you on the road. A GOOD bike from a bike shop will start about $250-500, the differences from a $59 Wal-Mart bike are many, they will last longer, work better, need adjustments less frequently, and be more fun to ride... but for now, just get started with whatever you can afford (you will not regret spending on quality). A good bike will be good for at least 20 years.

There are only a few things you need to take care of, keep air in the tires, a little oil on the chain, and adjust brakes and seat so it fits you right and everything moves without rubbing (you AND the bike). If you can't get anyone to help you and you're not mechanical, take it in to your nearest bicycle shop. Bike shops are run by people that love bikes and love to talk bikes, they will rarely try to sell you anything you don't need. They will understand your situation and give you good advice (don't be afraid to say you don't understand, please explain). If you want to ride your old bike at the lowest cost possible, let them know, and they'll treat you fair. When you get more serious and start thinking about a new bike, they will know where you've come from and steer you right. 

HELMET - a must ! 80% of serious bike injuries involve the head. Your odds of avoiding brain trauma are good, if you wear a helmet every time. Cheap is OK. The ONE DAY I didn't have a helmet on this year, going for a 1/2 mile ride to dinner, I crashed, on a concrete median, luckily I didn't hit my head. If you think you're too cool let me introduce you to a fella I work with who lost a couple teeth and had his helmet cracked on the way to work one day, oh, he missed a week of work and got the teeth replaced, but it could have been worse if he hadn't a helmet (oh and he was in a bike lane with the right of way, but that didn't matter to the SUV) !

TOOLS - a basic multi-tool, an air pump, a flat repair kit, tire levers, & spare tube. A little under the seat bag will hold it all, the whole kit will be $25-50. It's all you will need for a while. 

CLOTHES - should be loose and wick moisture (if you are in a warm climate, like Florida). Tennis shoes work fine, but they should give support, I find Converse and sandals cause my arches to hurt unless the pedals are big and flat.

START RIDING - START SMALL & HAVE A DESTINATION ! The thing I've found with beginners is, if you have fun, you'll want do it more. In the beginning, 1 to 2 miles (5-10 minutes) each way is plenty enough. Try the grocery or drug store or a little errand to give your ride purpose. If it's a "fitness ride", start out with 5, 10, or 15 minutes, and build up a minute or two the next day! Your body is amazing, you will advance quickly, and in about a month you will be way stronger and faster than you started just by getting your muscles fit.

The biggest thing people do wrong is overdo it, I hear the stories all the time and "man am I sore"... I was having so much fun I kept going... (but didn't realize you have to double the distance to get back home). SO HAVE FUN, take short routes with detours or stop offs, and as you gain experience and fitness you will naturally go further with less effort. This is where a journal helps out, how many miles/minutes are you riding a day, a week, a month... recap at the end of the month, and also write when you got sore, where, and when you overdid it... you'll recognize it earlier the next time and learn to back off.

Group rides and clubs are good ways to get started too, they mostly have all levels of riders so you can get with a group going just your speed. Search "Orlando bike clubs" or your city or state... you'll find plenty.

After a little while you'll find you want to add something, maybe a computer or rear view mirror, or rack, grocery bag, whatever, that's when you'll be looking through some of the links for things to buy and find out how other people are doing it. Or you can just go in the shop. I can tell you from my 11 yrs experience I've bought a bunch of stuff that didn't work out right, thinking "if it doesn't work out I'll buy a good one later"... so you spend too much, hold off, take your time, and go for quality. You're going to be riding a long time so get your gear like it will last you forever.