I have ebay converters that make a 1999 CRT tv accept the RF adapter output of older systems like the Atari 400. My recent 400 purchase showed me RF video fine with internal speaker working and all keyboard buttons working beautifully. The audio however is very buzzy and thought someone here may know some tests or grounding. Down the road I may look into the mods to bring this up to 2018 standards but first wish to test and troubleshoot it in its natural state.
A video would really help to diagnose problems, but audio buzzing is often a ground loop. Try to use either the same household circuit to plug everything in, or different ones, in order to see if that might be part of the issue.
If that doesn't make a difference, unplug the Atari, spray some electrical contact cleaner into the channel select slide switch on the 400 and then cycle it vigorously about a half dozen times, then wait a few minutes for the cleaner to dry. Sometimes those switches get corrosion or dirt inside them and cause glitches in the signal.
If it's not either of these things, you're probably gonna have to open things up to take a look. Of course, this presumes your RF converter isn't the cause of the issue. Not sure if you're referring to a physical demodulator box or just an RCA-to-COAX adapter that lets you attach the RF signal to the CABLE/ANT input on your TV. Some more details on the equipment might help figure out what's going on as well.
Anyway, if you take it apart, I strongly suggest you hit up AtariMania or Archive.org and download the Atari 400-800 Service Manual or the SAM'S COMPUTERFACTS. Both of them have troubleshooting steps, schematics and diagrams, etc.