1. I'd get the 130XE over the 800XL, its the most compatible with the entire library, doesn't have as good of a keyboard as the 800XL, and I don't think it looks as nice, but if you are just wanting a box for your game room with a cart full of games on it, the 130xe is going to be the most compatible.
2. You can use standard Atari joysticks and paddles
3. 5200 roms will not run on it but by now almost all the roms for the 5200 have 8-bit conversions and you can run any of those.
4. Original Atari PSU's are usually fine (there is an 'ingot' variety that is known to fail and kill Atari's but all the rest are fine). If you get an Atari with an ingot, cut the cables off it for spares and throw it away.
5. If you want compatibility with modern games and old games released later in the Atari's life, you might want to think about getting a PAL Atari over an NTSC one as most of the modern software is written in Europe, NTSC tends to be an afterthought or last step in the dev process. This will mean you also need a display you can use with a PAL machine.
I deliberately didn't answer the 'which cart' question, there are lots of choices and _everyone_ has an opinion and favorite. Mine is the SUB Cart or AVG Cart, both will work for you. Other people will be along shortly to recommend all the other ones!