While I have made some progress, I've also encountered a bunch of frustration.

The progress is mostly on the hardware side where I realized I could plug the Raspberry Pi back into the monitor's USB hub with a old USB A to B cable via the USB B jack to micro USB OTG cable.  Then I had the second realization I could simply plug in a pair of USB headphones for audio (and use cheap USB speakers in the final build).  While the USB speakers won't be as loud as ones powered by the 12V jack on the monitor - I wasn't finding any cheap 12V amplifiers for the RPi.

The software, however, has been a mess of frustration.  The idea of this project is to rotate the monitor and configure it to play vertical arcade games.  You'd think I wouldn't be the first person to have this idea and it would be a simple configuration option.  While MAME has the support built-in, neither Lakka nor the front end used by other prebuilt system images (EmulationStation) include this function.  Attract-Mode, another front-end available for the RPi, has this function - but it's offered as an add-on to RetroPie rather than as a dedicated system image.  The other option is to rotate the display via a configuration option, but I've heard that has a significant performance impact.

For the Raspberry Pi there are several prebuilt system images based around the Libretro emulation library:

1. Lakka - uses the RetroArch front-end which looks a lot like the Sony XMB interface.  No support (AFAIK) for rotation.  Some quirks (e.g. image sets the HDMI option to CEA (TVs) which causes problems with monitors (DMT), conflicts between Colecovision and MSX emulators).
2. Recalbox - uses the EmulationStation front-end which is the typical list + image view.  No support (AFAIK) for rotation.  Basic image has a 3GB FAT32 partition but only 756MB of data which then causes the initial setup to fail on a 4GB microSD card.  Resizing the partition to 800MB works around the problem.  On the plus side it includes a bunch of games with non-commercial licenses and has some nice menu music.
3. RetroPie - uses EmulationStation by default, but has the option to install Attract-Mode.  MAME4ALL doesn't work on most recent version with non-HDMI audio, okay with older version.  Haven't figured out how to get USB audio working.  (Although maybe I can get the necessary info from the Recalbox image...)

Update - turns out EmulationStation can be rotated via a command line option.  Unfortunately Recalbox runs from a read-only filesystem so I can't add it.  So my current focus is on getting USB audio working on RetroPie.