As ijor already mentioned latency is the big issue, and we'll have latency in the (USB/serial) driver code, the USB controller/protocol and the USB-serial adapter.
A couple of months ago I did some measurements with FTDI chips on Linux, with the serial driver configured to low-latency mode (to avoid delays when sending data, plus IIRC that also configures the FTDI chip to use the lowest latency timer setting of 1ms).
On a USB3 (XHCI) controller or on a USB2 (EHCI) controller with an USB2-hub inbetween (so the EHCI controller doesn't route the USB traffic to one of the UHCI or OHCI companion controllers) the minimum roundtrip latency was near the 125us microframe interval.
On an UHCI controller it was near the standard 1ms frame interval, on an OHCI controller it was twice that - not sure if that is an issue with the OHCI controller or the Linux driver.
In addition to that the FTDI chip won't send data packets faster than 1ms (the minimum latency timer setting). So, for example, when you receive a command frame ACK plus COMPLETE byte from a peripheral these two bytes (which are specced to have a gap of at least 250us between them) will be received as a single packet with 2 bytes by the PC. While that's no big deal for sector reads (which often take a long time) other commands like "get status" will have about that minimum gap.
That alone is a nasty thing, as when talking to a XF551 you should switch from 19200 bit/sec to 38400 bit/sec immerdiately after the ACK, before the COMPLETE
When acting as a SIO device you have all the time you want before switching speed and sending the complete on Read commands, but on write commands the Atari will start sending data (at high speed) about 1ms after it received the command frame ACK.
You can imagine that you won't have much chance on a full speed USB connection with 1ms frame interval (you also need to repeatedly query the USB adapter if it has finished transmitting the byte), and even with a 125us interval it's rather hard to get the timing right - other devices on the USB bus may be blocking it (so you're at 250-500us round trip time) and/or latencies in the OS/driver will ruin the timing.
With "real" serial ports things already look a lot better. There you don't have the latencies from the USB bus and if talking to the chip directly (bypassing the stanard drivers from the OS, like what I do with the atarisio kernel driver for 16550/16C950 UARTs) latencies can be reduced a lot and the XF551 protocol is no problem.
OTOH with other USB-serial adapters like the prolific 2303 things are even worse, they (or at least the linux drivers) don't seem to support querying the transmitter empty status - so one has to "guess" if transmission is actually finished and wait blindly for a rather long time until it's safe to switch serial speed.
Edited by HiassofT, Sat Apr 20, 2019 11:57 AM.