I recently dug out my old 2600 and decided to check the power supply with my multimeter before plugging it into the system. When I checked the voltage I was getting somewhere around 13v, while the label on the power supply states it's only supposed to put out 9v. Is this normal?
Yes. The rated voltage is a measure of what the power brick produces under load.
EDIT: To prolong the life of the power adapter, always plug it into the system before you plug it into the wall, and always unplug it from the wall before you unplug it from the system. That's due to the 1/8 inch mono plug they used on it.