I'll give the retrovgs people credit, they don't give up on trying to get this thing made. They really seem to believe in it. They're the only ones but hey.
First, ditch that controller. If that's the controller they're going to include, just don't include a controller at all. Make it so that you can interface with any generic bluetooth controller like the 8bitdo controllers. Or at least let it work with a bluetooth dongle add on. USB is not enough.
If it's a retro system, you don't need an analog stick. You just don't. why not add gyro's while you're at it then.
I'd make the first target system for the FPGA something like the Gameboy Advance. It's relatively recent enough that companies can port their old GBA IP's to this thing, and maybe make use of a 4 face button controller. If they want to add future features through firmware upgrades, that's fine too.
Finally, I'd include some sort of online component to it. 32 bit games with online multiplayer could be interesting.
But that's just me.