Using serial connection on LTO Flash!

[ppelleti]

The documentation for LTO Flash! suggests that it is possible for a program on the Intellivision to establish a serial connection with the host computer over USB. Does anyone know the details for how to do this?

 

My current motivation is that I've written a simple bitmap editor that runs on the Intellivision, and I thought it would be handy to have it be able to print the finished bitmap over the serial connection to the host computer, to make it easier to incorporate into a program. But I can think of lots of other things that the serial connection would be useful for, too.

 

[intvnut]

Yes, it absolutely is possible to establish a serial connection from software on the Intellivision to the host.

 

On the Intellivision side, I tried to keep it as simple as possible, while still being somewhat efficient. There's three memory locations involved:

• $0F0F: USB sense. Returns 1 if power is applied on USB.
• $0F10: UART RX. Returns non-zero if data is available. Data in lower 8 bits.
• $0F11: UART TX. Returns zero on read if it's safe to TX. Transmits the lower 8 bits on a write.

Very simple serial driver:

.

;; ======================================================================== ;;
;;  Very Simple Serial Driver for LTO Flash                                 ;;
;; ======================================================================== ;;

LTO_UART        PROC
@@usb_sense     EQU     $F0F        ; 0 = USB off; 1 = USB on
@@rx            EQU     $F10        ; Data in bits 7:0, status in bits 15:8
@@tx            EQU     $F11        ; Reading 0 means safe to TX.
@@baud          EQU     $F12        ; Read-only

                ;; Status bits for RX
@@stat.rxd      EQU     $0100       ; RX data available
@@stat.oerr     EQU     $0200       ; RX overflow error
@@stat.ferr     EQU     $0400       ; RX framing error
@@stat.perr     EQU     $0800       ; RX parity error
                ENDP    

;; ======================================================================== ;;
;;  LTO_XMIT    -- Send byte in R0                                          ;;
;; ======================================================================== ;;
LTO_XMIT        PROC
                CLRR    R1

@@wait_tx:      CMP     LTO_UART.tx,    R1
                BNEQ    @@wait_tx

                MVO     R0,    LTO_UART.tx
                JR      R5
                ENDP
                
;; ======================================================================== ;;
;;  LTO_RECV    -- Receive byte in R0.  Status in MSB of R0.                ;;
;; ======================================================================== ;;
LTO_RECV        PROC
                CLRR    R0
@@recv_loop     ADD     LTO_UART.rx,    R0
                BEQ     @@recv_loop
                
                JR      R5
                ENDP    
                

.

In IntyBASIC, you can use PEEK and POKE. I haven't tested the following, but it should work:

.

CONST LTO_usb   = $0F0F
CONST LTO_rx    = $0F10
CONST LTO_tx    = $0F11
CONST STAT_rxd  = $0100
CONST STAT_oerr = $0200
CONST STAT_ferr = $0400
CONST STAT_perr = $0800

' Send the data in CH when it's safe to do so
UartTx: Procedure
    While Peek(LTO_tx)
        If Peek(LTO_usb) = 0 Then Return
    Wend
    Poke LTO_tx, CH
End

' Receive data into CH, with status in ST.
' Uses #T as a temporary.
UartRx: Procedure
    Do
        If Peek(LTO_usb) = 0 Then Return
        #T = Peek(LTO_rx)
    Loop Until #T <> 0
    CH = #T
    ST = #T / 256
End

.

On the host side of things, that's where you get into serial programming. The UART on LTO Flash shows up as a bog-standard FTDI serial adaptor. You need to configure it for 2000000, 8-N-1. I know the POSIX way to talk to serial ports, but I'm not actually the expert here. I may need to pull intvsteve in for some tips on serial programming on the host side.

 

Note that while I did include status bits for parity, framing, and overflow, in practice those never really occur unless you yank a cable or kill the software on the host or misconfigure baud rates. (LTO Flash's MCU is fixed at 2Mbps, while the FTDI chip is actually programmable.) Once you're configured for 2000000bps, 8-N-1, your software can pretty much ignore the status bits other than "rxd" which says "data's available."

 

 

Now, another option is to use jzIntv's Emu-Link File I/O (ELFI) support. Intellivision programs can read and write files on the hard drive if you let them. If you're curious about this feature, let me know. There is a "fileio" example distributed with jzIntv. I haven't translated it to IntyBASIC though.

[ppelleti]

Thanks for the detailed information! I will give that a try. I've done a bit of stuff with serial before, so I think I'll be okay on the host side.

[+intvsteve]

Somewhere in the ever-expanding pile of projects to do someday, I jotted down an idea of a general 'plugin' (via C#) interface for LUI to just act as a conduit for steering the serial stream to the plugin. I.e. LUI would leverage its existing port identification and configuration and device validation, letting the plugin provide the 'business logic.' The desktop UI would discover plugins software that you could associate with a ROM, and the UI would 'hand off' serial port communication to said plugin while a game was running. The super-slick version would need a little handshake between the cart and the UI so the UI could know *which* ROM you were launching. On the other hand, if you launched a ROM directly from the UI, experimentation on such a scheme could start w/o any firmware tweaks.

 

That's all uber complex though, and literally nothing more than a twinkling of an idea in my mind's eye. If it's fast proof-of-concept you're interested in, then writing something bespoke for your project on the host would be entirely under your control.

 

To get host-side (that is, Mac/Windows/Linux) serial port communication working, the trickiest part is whether your serial API supports the baud rate. C/C++ likely works pretty much "out of the box". It was a little more ... adventuresome getting C# going, but it's doable.

 

Another quick-and-dirty trick you might try for your application:

Instead of dumping the bitmap out the serial port, save it on the cart's flash memory directly. While a little cumbersome, you *could* then do a 'cart backup' and locate the data in the backup. I'd need to point you out to the trick of finding the file on your computer, but it's not too terrible.

[intvnut]

Somewhere in the ever-expanding pile of projects to do someday, I jotted down an idea of a general 'plugin' (via C#) interface for LUI to just act as a conduit for steering the serial stream to the plugin. I.e. LUI would leverage its existing port identification and configuration and device validation, letting the plugin provide the 'business logic.' The desktop UI would discover plugins software that you could associate with a ROM, and the UI would 'hand off' serial port communication to said plugin while a game was running. The super-slick version would need a little handshake between the cart and the UI so the UI could know *which* ROM you were launching. On the other hand, if you launched a ROM directly from the UI, experimentation on such a scheme could start w/o any firmware tweaks.

 

Wow, yeah, that sounds super complicated. :-) Slick if it works, but a lot of moving parts.

 

The main issue I ran into on the host side in MacOS was being able to specify the baud rate. I ended up having to alias 300bps to mean 2000000bps for my cruddy C code that used the basic POSIX API. intvsteve had better luck with a more native API as I recall.

 

On Linux, I remember it being much more straightforward.

 

And Windows? I have zero clue on Windows.

[+intvsteve]

I did a couple experiments a few years back in Windows but specifics escape me now. In one case, I may have been trying with POSIX APIs in MSYS... Then there are the quasi-POSIX-like APIs supported in Windows... I wonder if the various Linux subsystems you can run in Windows 10 now would be up to the task?

 

The Windows experiment I did may well have started from this tutorial...

 

On Mac I also ran into the baud rate issue from both the C and the C# side (which just was wrapping C API calls) and had to implement a workaround, too... I have the source for the C# trick (obviously), which would be easy enough to replicate in C.

 

But if you're already comfortable w/ serial programming, ppelleti, then ... make it so! It'll be cool to see what you do!

 

--

The slick-and-complicated idea goes something like this (in the C# world):

• Drop a C# assembly that exports an interface into plugins directory (LUI already actually has a very lightly tested capability here) - Interface TBD
• LUI populates a list of discovered plugins when it launches
• If you launch a game directly from LUI to LTO Flash! the 'monitor' in LUI hands off the serial port to the appropriate plugin
• When plugin is done, it cedes control back to LUI

OK, sure, in reality it's got to deal with all those icky error conditions and whatnot. But with the beefed up metadata support for ROMs - both in the SDK-1600 stack as well as in LUI, at least part of the 'mapping' problem should be relatively simple to specify.

[intvnut]

If you just want to stay within jzIntv and use its file I/O facilities, and you're comfortable programming assembly language, there's a set of macros in jzintv/examples/macro/el_fileio.mac that will get you started. jzIntv supports the following file operations:

.

;; ======================================================================== ;;
;;  Emu-Link Support Macros:  File I/O                                      ;;
;;                                                                          ;;
;;  The File I/O API provides a traditional C-style file I/O API to the     ;;
;;  Intellivision.  jzIntv sandboxes the file I/O to a single directory     ;;
;;  specified on jzIntv's command line.                                     ;;
;;                                                                          ;;
;;  Filenames can be no longer than 31 characters, and are limited to       ;;
;;  alphanumerics, "-", "_" and ".".  The filenames may not start with      ;;
;;  a ".", either.                                                          ;;
;;                                                                          ;;
;;  The file I/O API allows up to 8 files to be open simultaneously, as     ;;
;;  long as jzIntv's host OS also permits.                                  ;;
;;                                                                          ;;
;;  On platforms that distinguish between binary and text files, jzIntv     ;;
;;  attempts to open files in binary mode.                                  ;;
;;                                                                          ;;
;;  API SUMMARY                                                             ;;
;;                                                                          ;;
;;      ELFI_OPEN       Opens a file, returning a file descriptor           ;;
;;      ELFI_CLOSE      Closes a file descriptor                            ;;
;;      ELFI_READ       Reads bytes from a file                             ;;
;;      ELFI_READ16     Reads words (big-endian format) from a file         ;;
;;      ELFI_WRITE      Writes bytes to a file                              ;;
;;      ELFI_WRITE16    Writes words (big-endian format) to a file          ;;
;;      ELFI_LSEEK      Seeks within a file, returning new file offset      ;;
;;      ELFI_UNLINK     Unlinks (removes) a file.                           ;;
;;      ELFI_RENAME     Renames a file.                                     ;;
;;                                                                          ;;
;;  See below for details on the API.  The API macros attempt to take       ;;
;;  their arguments either in registers or as constants.  One exception:    ;;
;;  APIs that take a file descriptor require you to *always* pass it in     ;;
;;  register R2.  For APIs that take filenames, give a pointer to the       ;;
;;  ASCIIZ string, not the string itself.                                   ;;
;;                                                                          ;;
;;  The API documentation below comes directly from jzIntv.  If you         ;;
;;  choose to pass in some arguments as registers, you must match the       ;;
;;  indicated register assignments.  See special note at LSEEK.             ;;
;;                                                                          ;;
;; ======================================================================== ;;

.

You will need to invoke jzintv with the --file-io flag to enable this feature. It's not enabled by default, as I didn't want jzIntv to become an attack vector for malicious games. :-)

 

It should be possible to make IntyBASIC wrappers around these APIs as well if someone wants to make some file I/O routines for IntyBASIC.

 

jzIntv actually has a number of Emu-Link APIs built around a common mechanism (documented in jzintv/examples/macro/emu_link.mac). It (ab)uses the SIN instruction to interface with the emulator. You load certain registers with values, and the emulator responds accordingly. The Emu-Link API is what allows the event_diag.rom to function, actually. There are currently Emu-Link APIs for collecting raw events, raw joystick data, and file I/O. IntyBASIC wrappers around this would just need a touch of inline assembly, or get structured as CALL-able assembly code.

 

Also, building on intvsteve's recommendation here:

 

Another quick-and-dirty trick you might try for your application:

Instead of dumping the bitmap out the serial port, save it on the cart's flash memory directly. While a little cumbersome, you *could* then do a 'cart backup' and locate the data in the backup. I'd need to point you out to the trick of finding the file on your computer, but it's not too terrible.

 

If you happen to do that within jzIntv and specify a --jlp-savegame file, those bits will be written directly to disk. The format of that file is unsurprising, and should be identical to what you'd get from an LTO Flash backup. While it wasn't an original design goal, I did try to keep all these pieces compatible and straightforward.

 

So, you can use the FLASH.xxx facilities in IntyBASIC (compiled with the --jlp flag) as another way to get data in and out of an IntyBASIC program.

[ppelleti]

Apparently this isn't as easy as I thought, although I'm not sure what I'm doing wrong.

 

I wrote a simple program (serial.bas in the zipfile below) which continuously prints the letter "x" to the serial port, as long as the USB is connected.

 

First, I tried connecting using the "screen" command that comes with OS X:

screen /dev/tty.usbserial-JZJKUYAG 2000000

 

and that didn't work (nothing was printed). So then I wrote a program for the host side (show-serial.c in the zipfile below) and tried that:

./show-serial /dev/cu.usbserial-JZJKUYAG

but that didn't print anything either.

 

All my testing so far has been on Mac OS X 10.9, but I will try Windows next. Unfortunately, my Linux computer is in another room and it's not particularly easy to move, so it's difficult for me to test on Linux at the moment.

 

I will definitely give Emu-Link a try, so I can support the simulator, but I'd like to be able to run on real hardware, too. (And although using flash sounds like a good workaround, I'd ultimately like to figure out what I'm doing wrong with the serial port.)

 

usb-serial.zip

[intvnut]

A few quick things:

• Be sure to disconnect from LTO Flash in the Mac GUI. ⌘D is the keyboard shortcut on Mac.
• If you leave LTO Flash at its title screen, it'll repeatedly print out "LOCUTUS\n", about once per second. If you can see that 'beacon' then you're seeing the serial port. If you don't see that beacon, you're not seeing the serial port.
• I've never managed to get 'screen' to connect at anything faster than 115200 or the like. IIRC, minicom worked for me, and now I use https://www.decisivetactics.com/products/serial/

 

My own serial code on the Mac and Linux is not very different from yours. I do open mine non-blocking, with no controlling TTY, and some other minor differences.

.

#ifdef MAC_SERIAL
#   define SPEED (B38400)
#   define MAC_SPEED (2000000)
#else
#   define SPEED (B2000000)
#endif

int open_serial
(   
    const char *const serial_fname
)
{
    const char *const device = serial_fname;

    struct termios tio; // Eew... termios!  Ah well...
    int fd;

    serial_fd = fd = open( device, O_RDWR | O_NONBLOCK | O_NOCTTY | O_NDELAY );

    if (fd < 0)
    {   
        perror("open()");
        fprintf(stderr, "Could not open serial device \"%s\".\n", device);
        exit(1);
    }

    if (ioctl(fd, TIOCEXCL) == -1)
    {   
        perror("ioctl()");
        fprintf(stderr, "Error setting TIOCEXCL on serial device \"%s\".\n",
                device);
        exit(1);
    }

    tcgetattr(fd, &tio);
    cfmakeraw(&tio);

    tio.c_cflag &= ~( CSIZE | PARENB );
    tio.c_cflag |= ( CLOCAL | CRTSCTS | CS8 | HUPCL );

    tio.c_oflag &= ~( OPOST | OLCUC | ONLCR | OCRNL | ONOCR | ONLRET | OFILL |
                      NLDLY | CRDLY | TABDLY | BSDLY | VTDLY | FFDLY );
    tio.c_oflag |= NL0 | CR0 | TAB0 | BS0 | VT0 | FF0;

    tio.c_iflag |= IGNBRK;

    tio.c_lflag &= ~( ICANON );
    tio.c_lflag &= ~( ECHO | ECHOE | ECHOK | ECHONL | ECHOCTL | ECHOPRT );

    tio.c_cc[ VTIME ] = 1;
    tio.c_cc[ VMIN  ] = 0;

    if (tcsetattr(fd, TCSANOW, &tio))
    {
        perror("tcsetattr()");
        fprintf(stderr, "Could not control serial device \"%s\".\n", device);
        exit(1);
    }

    if (cfsetispeed(&tio, SPEED))
    {
        perror("cfsetispeed()");
        fprintf(stderr, "Could not control serial device \"%s\".\n", device);
        exit(1);
    }

    if (cfsetospeed(&tio, SPEED))
    {
        perror("cfsetispeed()");
        fprintf(stderr, "Could not control serial device \"%s\".\n", device);
        exit(1);
    }

    if (tcsetattr(fd, TCSANOW, &tio))
    {
        perror("tcsetattr() b");
        fprintf(stderr, "Could not control serial device \"%s\".\n", device);
        exit(1);
    }

#ifdef MAC_SERIAL
    {
        speed_t speed = MAC_SPEED;

        if ( ioctl( fd, IOSSIOSPEED, &speed ) == -1 )
        {
            perror("ioctl()");
            fprintf(stderr, "Could not control serial device \"%s\".\n",
                    device);
            exit(1);
        }

        // Set the receive latency in microseconds. 
        // Serial drivers use this value to determine how often to
        // dequeue characters received by the hardware. 
        // Most applications don't need to set this value: if an
        // app reads lines of characters, the app can't do anything 
        // until the line termination character has been received
        // anyway. The most common applications which are sensitive
        // to read latency are MIDI and IrDA applications.

        unsigned long mics = 1UL;   // 1 microsecond
        if (ioctl(fd, IOSSDATALAT, &mics) == -1) {
            perror("ioctl()");
            fprintf(stderr, "Error setting read latency for \"%s\".\n", device);
            exit(1);
        }

    }
#endif

    tcflush(fd, TCIOFLUSH);
    tcflow(fd,  TCOON);
    tcflow(fd,  TCION);
    return 0;
}

[ppelleti]

Thanks! I now have it working!

 

First, I tried minicom with LTO Flash! at the title screen:

 

minicom -b 2000000 -D /dev/tty.usbserial-JZJKUYAG

 

and didn't get anything. But then I downloaded the Decisive Tactics serial program you mentioned, started the free trial period, and I successfully got "LOCUTUS" as expected.

 

Turns out my serial.bas program had a small bug in it (using "<> 0" when I should have been using "= 0"), but once I fixed that, it printed out its infinite string of "x" on the Decisive Tactics terminal as expected.

 

I will try your serial code next, and see if that works better than "my" serial code. (Which is not really mine; I borrowed it from an MIT-licensed library I've used before.)

 

[intvnut]

BTW, you've reminded me that I need to finish writing the Locutus Wire Protocol document I started awhile back. I'm basically translating all of my design documents for LTO Flash into Google Docs that are a little more friendly to read.

 

(Yes, I wrote copious design docs. Ask intvsteve! I also wrote multiple emulations of various aspects of LTO Flash before I wrote the firmware. And yes, I will publish all this.)

[ppelleti]

Oh, it turns out my show-serial.c does work. The problem is just that I was testing it against the version of serial.bas that didn't work. But show-serial.c works fine with LOCUTUS and with the fixed version of serial.bas.

 

However, I would like to fix my program so it doesn't busy-wait. Currently, when I get end-of-file, I sleep for a millisecond and then try again:

 

    for ( ; ; ) {
        ret = read (fd, &c, sizeof (c));
        if (ret == 1) {
            fputc (c, stdout);
            fflush (stdout);
        } else if (ret < 0) {
            perror (dev);
            close (fd);
            return EXIT_FAILURE;
        } else {
            usleep(1000);
        }
    }

Is there a way to get it to block until the next character, rather than returning EOF?

[intvnut]

Is there a way to get it to block until the next character, rather than returning EOF?

 

In my own code, I use an old-school select() loop to watch for the fd associated with the serial port to become ready, since I did indeed open the serial port to be non-blocking. I used a timeout of 1 second. The task will wake up as soon as a byte's available though:

.

int serial_recv_byte( void )
{
    char    c = -1;
    ssize_t r;
    int     ready, i;

    if ( serial_fd < 0 )
    {
        fprintf( stderr, "serial_recv_byte: device not open\n" );
        return -1;
    }

    for ( i = 0; i < TIMEOUT; i++ )
    {
        struct timeval timeout = { 1, 0 };
        fd_set serial_fd_set;
        FD_ZERO( &serial_fd_set );
        FD_SET ( serial_fd, &serial_fd_set );

        ready = select( serial_fd + 1, &serial_fd_set, NULL, NULL, &timeout );

        if ( !ready )
            continue;

        errno = 0;
        if ( ( r = read( serial_fd, &c, 1 ) ) == 1 )
            return c & 0xFF;

        if ( errno != 0 &&
             errno != EINTR && errno != EAGAIN && errno != EWOULDBLOCK )
        {
            perror( "serial_recv_byte: read()" );
            return -1;
        }

        if ( die )
        {
            fprintf( stderr, "serial_recv_byte: dying\n") ;
            return -1;
        }
    }

    fprintf( stderr, "serial_recv_byte: timeout\n" );
    return -1;
}

.

You could dispense with the TIMEOUT and have it loop forever. I personally set the TIMEOUT to 2400, so that corresponds to 40 minutes or something like that. Supposedly, you could pass in a NULL pointer for the timeout instead, and it'll wait "indefinitely." You still need to check whether it returned early and loop.

 

BTW, one word of caution on the Mac: If you open the serial device, do keep it drained. I discovered when testing LTO Flash that if I opened the serial device, but let the "LOCUTUS" beacons build up too much, eventually I'd mess up Apple's driver layer and lose USB. I could only recover with a reboot. The beacons aren't a problem if the device isn't open.

 

What's supposed to happen is that the FTDI driver is supposed to drop ~CTS to my MCU, and I stop sending. (And I promise, I look at ~CTS and honor it.) But on Mac only, I have that odd problem. I never saw this with Windows or Linux.

 

In your use case, I doubt this would ever be a problem. Your BASIC program and your command line tool are all that will be talking over serial, and it sounds like your command line tool will be ready to receive whatever needs to be received. In my case, I have a command-line tool I used for testing LTO Flash, and it has an interactive command line. It interacts with that LOCUTUS beacon mode. The interactive command line blocks, and so I don't drain the beacons. If I let it sit there for 30 - 40 minutes with the device open, I have a problem.

[ppelleti]

I got the EmuLink file I/O working. Here is an example of how to do it from IntyBASIC.

 

The *.mac files are from the jzintv distribution. The fileio.asm file contains assembly language wrappers I wrote for a few of the file I/O calls. (I only wrote wrappers for the calls I needed, so it is not complete.) The program file-test.bas calls the wrappers in fiieio.asm from IntyBASIC.

 

I invoke jzintv like this:

 

jzintv -z3 --file-io . file-test.rom

 

Then, just press any of the side buttons (for example, the shift key) and it will create the file testfile.txt if needed, and then append an "x" and a newline to that file.

 

Thanks for all of your help with this!

 

file-test.zip

[ppelleti]

I finished my bitmap editor, which now supports saving to serial on LTO Flash!, or saving to a file on jzIntv.

 

Source code is here:

 

https://github.com/ppelleti/BitmapEditor

 

and binaries are here:

 

https://github.com/ppelleti/BitmapEditor/releases