Depends what you're going for I suppose. Do you have to have that Voodoo instead of a cheap Nvidia card and nGlide? Must you have access to Roland music or the fastest cpu for your particular chipset? Like you say, if you want perfection you'll pay for it.
I think in general you can still pick up an old desktop and parts pretty cheaply that will play most games with a bit of tweaking.
Other computers? I dunno, they've all certainly raised in value over the past ten years but I guess that's to be expected. At the very least the C64 is still affordable here in the UK which is great since that's the computer I plan to pick up next. The Apple II GS however...