My $.02:
* Does it matter which model of the hardware I get?
Depends on what you're after. If you just want to play the games, then not really. If you care about "rarity" or "collector value", etc. then there are variations of the system that are sought after for those things. My opinion is that the units with the 6 switches are the nicest to use. The hardware section of this website can give you a crash course on what the different models are.
* Do I need a paddle controller and a joystick? Two of each?
You definitely need to get a good set of paddles to get the full 2600 experience. They come in pairs, so one set plugs into one input. If you plug them in and the controls are shaky on the screen, you need to clean the pot in the paddles. Plenty of info on the net on how that's done. Two joysticks recommended as well, so you can play two player games.
* Where is a good place to get it? eBay? Craigslist?
I've never bought Atari stuff on eBay because I find the prices atrocious most of the time. I'd recommend at least checking your local classifieds, thrift shops, video game stores, etc. I think it's becoming increasingly more rare to find good Atari deals using this route but if you find something you'll probably be able to save some cash (if for no other reason than you won't have to pay for shipping).
* What's a fair price for the hardware and some relatively common games?
I'd say $50 should be plenty to get you a fully working system, some controllers, and a few games.
* Is there anything I should be particularly wary of when buying a system?
The only thing I can think of is somebody trying to misrepresent which version of the system you're buying. For instance, make sure you know the difference between a "light sixer" and a "heavy sixer", the latter of which is more collectible.
* What should my first Atari game(s) be? (I've already seen the 100 best games thread)
This is a thread in itself, but to make a broad statement, start with some Activision titles.
* Does anybody out there want to sell/loan me what I need, by any chance?
Nope.
EDIT: There is one other thing I wanted to mention. This is personal preference, but IMO the best Atari 2600 games are games that were designed for the system as opposed to arcade ports. Not that there aren't some very decent "old school" arcade ports (Millipede, Missile Command, Ms. Pac-Man), but when they tried to bring some later games to the system I think the results were pretty crappy (Rampage, Double Dragon). Activision, which I mentioned earlier, has many great non-arcade-port titles (Frostbite, Pressure Cooker, Enduro, Keystone Kapers, Stampede, Megamania, and many more).
Edited by Cynicaster, Sat Apr 23, 2011 2:07 PM.