TIPI - TI-99/4A to Raspberry PI interface development

[+9640News]

5 minutes ago, jedimatt42 said:

So... we've talked about this before... here are the docs:  https://github.com/jedimatt42/tipi/wiki/PI.HTTP

 

It just doesn't work that way. And I'm not interested in making it do so.

Wasn't asking you to add/modify code.  Was trying to figure it out if it was supported or not, that's it.  I know there are concerns with security if it had been possible.

 

Thanks for the feedback.

 

Beery

 

[+jedimatt42]

If you want a secure way to have cloud storage on your TIPI, there are network filesystem drivers for linux, and you can mount a network drive as a folder under /home/tipi/tipi_disk/... 

 

At that point it is just more files and directories for TIPI... if someone pursued this and wrote up a howto... I'd be happy to link it into the wiki.

 

I believe I had whtech mounted with an FTP filesystem on the debut performance of TIPI... it showed that you need a network FS that does a heavy amount of caching.

[+InsaneMultitasker]

Had some unexpected play time this weekend. Thanks to M@ for answering a few questions along the way.

Pictures are the TIPI operating with my Geneve, in GPL mode, without the need for rompage.

Level 3 is about 95% operational.  I haven't sorted out how to deal with level 2. 

Don't even ask about native MDOS mode ...  

  

[GDMike]

Thx for that image. 

[+9640News]

14 hours ago, InsaneMultitasker said:

14 hours ago, InsaneMultitasker said:

Had some unexpected play time this weekend. Thanks to M@ for answering a few questions along the way.

Pictures are the TIPI operating with my Geneve, in GPL mode, without the need for rompage.

Level 3 is about 95% operational.  I haven't sorted out how to deal with level 2. 

Don't even ask about native MDOS mode ...  

  

Had some unexpected play time this weekend. Thanks to M@ for answering a few questions along the way.

Pictures are the TIPI operating with my Geneve, in GPL mode, without the need for rompage.

Level 3 is about 95% operational.  I haven't sorted out how to deal with level 2. 

Don't even ask about native MDOS mode ...  

   

Tim,

 

I had an issue in MDOS mode that I doubt translates into GPL mode.  I was not doing any file loading, rather just socket access so I do not know if the issue would present itself or not.  In MDOS mode, I made use of SNDMSG and RCVMSG routines for socket communications.  I think at first, this was with a Memex Geneve.  I found I was getting some erratic lockups.  It could have been a coding issue, however after I copied the code for those routines and used them in my own code, the issue disappeared.  I'm not sure if there was a timing issue with code running from the DSR versus the Memex memory or not.

 

The AfterHours BBS code I used the lifted routines as I had the memory space to do it.  Matt's telnet client uses the DSR code, so it must not be an issue at least on the GPL side of things.

 

Just something to keep in mind if something doesn't respond the way you think it should.  

 

Beery

[+InsaneMultitasker]

Thanks Beery. So far what i have learned is that watching the TIPI log is very useful to determine what is happening with file access/file IO. I was able to squash a few bugs in my own code that way.  Now I keep a PUTTY session open to watch the log in real time     I'll describe what I'm doing in another thread for the Geneve/TIPI code once I get further along.  Right now it is more proof of concept stage though I'm pretty confident most of the TIPI support can be handled within GPL while at the same time not losing access to the other peripherals the master DSR manages.

[aftyde]

Hi Team, wondering if there is a way to hack the code such that it reports 2280 or 5760 sectors free instead of the default 1440?  Cheers, Arthur...

[+jedimatt42]

40 minutes ago, aftyde said:

Hi Team, wondering if there is a way to hack the code such that it reports 2280 or 5760 sectors free instead of the default 1440?  Cheers, Arthur...

 

I could make that a setting... hacking has a tendency to make updates fail... 

 

What is the use case for that? sectors used and sectors free is just a lie so I'm open to improvement here... ( Oh, I see now over here: 

 

[+jedimatt42]

I had imagined here: https://github.com/jedimatt42/tipi/blob/8feed0f6cf74adef1670ec9d15d578fa9b483623/services/ti_files/CatalogFile.py#L80

and here: https://github.com/jedimatt42/tipi/blob/8feed0f6cf74adef1670ec9d15d578fa9b483623/services/ti_files/CatalogFile.py#L95

 

that the values for mapped drives might want to be different than the values for TIPI. device... 

 

I think I saw HRD gets away with much larger values. 

 

I could let you OPEN #1:"PI.CONFIG"::PRINT #1:"SECTOR_COUNT=9900"::CLOSE #1

 

and that would be persisted, and 9900 or whatever you put there would be the value... What are the limits to this? I believe 16bits are used to refer to sector numbers or block numbers within a file... so 0-64k? a.k.a unsigned int?

[+Lee Stewart]

2 hours ago, aftyde said:

Hi Team, wondering if there is a way to hack the code such that it reports 2280 or 5760 sectors free instead of the default 1440?  Cheers, Arthur...

 

The standard TI DSR expects a disk’s VIB to have a sector allocation bitmap of 200 bytes (1600 bits) at the end of the VIB for a maximum of 1600 sectors, which is why the nanoPEB and CF7+ were set up with that many sectors/volume. I do not know how hard disk DSRs were set up to handle greater capacity (@InsaneMultitasker, @mizapf, @Tursi, @Ksarul, @atrax27407, @acadiel ...?), but any that were patterned after the TI DSR, I would suspect, are set up with a 1-sector “VIB” in each subdirectory that would limit each subdirectory to a similar, 1600-sector maximum. If this is so, the maximum size of a single file would be 1597 sectors, which is the maximum volume size (1600) minus the VIB, FDIR and FDR (for only 1 file) sectors. The size of a file is further limited to 32767 records, which should never be a problem unless record size is 12 bytes or less!

 

...lee

[+InsaneMultitasker]

@Lee Stewart  It sounds like M@ has a feature to take care of the issue, so long as the program doesn't use sector IO to compute the space    To somewhat answer your capacity question...

for strictly floppy-style formats, total (and max) capacity is computed by the available bitmap bits and the number of sectors each bit represents.  At one sector per bit (or allocation unit) max capacity is 1600 sectors.  2 sectors/AU = 3200 sectors.  4/AU and 8/AU give you 6400 and 12800 sectors, respectively. There is a caveat in that the cluster chain pointers (held in each file's FDR) are limited to 12 bits each so that when your capacity exceeds 4096 sectors, the cluster nibbles must now represent more than one sector.   The calculations here are a little more complex and is the primary reason the HFDC in a TI system cannot properly use High Density disk capacity, even though the controller hardware supports it.   Hard drive formats use a 31 sector bitmap with similar sector/AU calculations and more forgiving cluster chains. 

 

[+jedimatt42]

59 minutes ago, Lee Stewart said:

 

The standard TI DSR expects a disk’s VIB to have a sector allocation bitmap of 200 bytes (1600 bits) at the end of the VIB for a maximum of 1600 sectors, which is why the nanoPEB and CF7+ were set up with that many sectors/volume. I do not know how hard disk DSRs were set up to handle greater capacity (@InsaneMultitasker, @mizapf, @Tursi, @Ksarul, @atrax27407, @acadiel ...?), but any that were patterned after the TI DSR, I would suspect, are set up with a 1-sector “VIB” in each subdirectory that would limit each subdirectory to a similar, 1600-sector maximum. If this is so, the maximum size of a single file would be 1597 sectors, which is the maximum volume size (1600) minus the VIB, FDIR and FDR (for only 1 file) sectors. The size of a file is further limited to 32767 records, which should never be a problem unless record size is 12 bytes or less!

 

...lee

 

But I'm not emulating the internal structure of a spinny thing. Just trying to lie to old software in a way that doesn't anger it.

 

With a record count limitation, ( I see that in the api spec now ) that leaves the possibility of 32767 blocks in a file, each with one 254 byte record. 

For direct file access, you get 16bits to refer to logical block numbers within the file. 

For SCI, WDS, IDE devices, you get an extra 16bits extension to that logical block number. I did not implement that extension for TIPI... 

 

So TIPI, can have 256 bytes * (2^16 blocks) == 16Megabyte files if not using record access (half that with records). And many of them... I didn't limit how many files you can have per directory... If you use software that cares, don't put so many files in the directories you use with that software..

 

 

 

[+jedimatt42]

Ah the code I stole from Tursi's FIAD proposal for encoding integers to ti floats, doesn't handle positive integers greater than 9999... so that'll be the limit I can express... 

[aftyde]

7 hours ago, jedimatt42 said:

I had imagined here: https://github.com/jedimatt42/tipi/blob/8feed0f6cf74adef1670ec9d15d578fa9b483623/services/ti_files/CatalogFile.py#L80

and here: https://github.com/jedimatt42/tipi/blob/8feed0f6cf74adef1670ec9d15d578fa9b483623/services/ti_files/CatalogFile.py#L95

 

that the values for mapped drives might want to be different than the values for TIPI. device... 

 

I think I saw HRD gets away with much larger values. 

 

I could let you OPEN #1:"PI.CONFIG"::PRINT #1:"SECTOR_COUNT=9900"::CLOSE #1

 

and that would be persisted, and 9900 or whatever you put there would be the value... What are the limits to this? I believe 16bits are used to refer to sector numbers or block numbers within a file... so 0-64k? a.k.a unsigned int?

 

That could work...  in the spirit of fooling ancient software, it might make sense to decrement the free blocks by the size of whatever files happen to be in the directory.  Or am I scope creeping...    Largest floppy I have ever gotten to work is 2880 sectors, DSDD 80 track.  On my 200MB SCSI drive - it reports a total size of 781376, 741024 free and 40352 used.

 

 

Edited May 20, 2020 by aftyde

[+jedimatt42]

1 hour ago, aftyde said:

 

That could work...  in the spirit of fooling ancient software, it might make sense to decrement the free blocks by the size of whatever files happen to be in the directory.  Or am I scope creeping...    Largest floppy I have ever gotten to work is 2880 sectors, DSDD 80 track.  On my 200MB SCSI drive - it reports a total size of 781376, 741024 free and 40352 used.

 

 

 

I think it would be pretty poor of software to care how much is used, except to figure out how much is free... and on a TIPI, you have gigabytes free. So it seems to me that the lie of only 2 sectors being used, is a good middle ground.

[+9640News]

I've never been well versed on all the DSR capability on the TI-99/4A side of things.  So, with that being said, I would not want to see the TIPI DSR reporting anything back to the various calls that would prevent a file transfer via an Xmodem transfer of any size and/or number of files at the destination directory.

 

Beery

[Tursi]

On 5/19/2020 at 5:54 PM, jedimatt42 said:

Ah the code I stole from Tursi's FIAD proposal for encoding integers to ti floats, doesn't handle positive integers greater than 9999... so that'll be the limit I can express... 

Hehe, the sad part is that's a port of the code in the TI Disk Controller DSR.

 

[+jedimatt42]

3 hours ago, Tursi said:

Hehe, the sad part is that's a port of the code in the TI Disk Controller DSR.

 

That's good to know. It implies it is probably good enough for all integer ranges... Or any assembly code that is going to read them.  

 

What is the integer range limit as encoded in the this format? I only remember it is an 8 byte float, I think... 

[+jedimatt42]

http://www.unige.ch/medecine/nouspikel/ti99/reals.htm

[+Lee Stewart]

8 hours ago, jedimatt42 said:

That's good to know. It implies it is probably good enough for all integer ranges... Or any assembly code that is going to read them.  

 

What is the integer range limit as encoded in the this format? I only remember it is an 8 byte float, I think... 

 

The largest integer that can be represented in the 7-byte mantissa of an 8-byte (radix-100) float is 99999999999999, each byte of the mantissa having a value of 99 (>63). 99999999999999 is represented as >4663 >6363 >6363 >6363.  Of course, this will not work in Assembly Language Code (ALC) for one-word integers, but the maximum signed integer (32767) can certainly be represented: >4203 >1B43 >0000 >0000.

 

Here is the routine (CIF) I use in fbForth 2.0 to convert a signed 16-bit integer to floating point using FAC (>834A) for both input and output:

Spoiler

*     _______  __     _       ____     __                     __     
*    / __/ _ \/ /    (_)     /  _/__  / /____ ___ ____ ____  / /____ 
*   / _// ___/ /__  _       _/ // _ \/ __/ -_) _ `/ -_) __/ / __/ _ \
*  /_/ /_/  /____/ (_)     /___/_//_/\__/\__/\_, /\__/_/    \__/\___/
*                                           /___/                    
*            ______          __  _             ___       _      __ 
*           / __/ /__  ___ _/ /_(_)__  ___ _  / _ \___  (_)__  / /_
*          / _// / _ \/ _ `/ __/ / _ \/ _ `/ / ___/ _ \/ / _ \/ __/
*         /_/ /_/\___/\_,_/\__/_/_//_/\_, / /_/   \___/_/_//_/\__/ 
*                                    /___/                         
*
*-------------------------------------------------------------------------
*           _____________
*          / ___/  _/ __/
*         / /___/ // _/  
*         \___/___/_/    
* 
*-------------------------------------------------------------------------
*       NAME:  CONVERT INTEGER TO FLOATING POINT NUMBER
*    VERSION:  5/25/87   BASE LINE (from MDOS L10 Floating Point Library)
*   MODIFIED:  08/12/2014 & 01/16/2016  Lee Stewart
*              05/22/2020 for this snippet  Lee Stewart
*
*      INPUT:  FAC = INTEGER to convert to Floating Point (2 bytes)
*
*     OUTPUT:  FAC = FLOATING-POINT INTEGER result (8 bytes)
*
*       CALL:  BL @CIF
*-------------------------------------------------------------------------
*
FAC    EQU  >834A

 

CIF    LI   R4,FAC         get address of number (INT in, FP out)
       LI   R6,FAC         for clearing (see below)
       MOV  *R4,R0         get the integer before we clobber it
*
       LI   R7,100         FOR SPEED [radix-100 (R100) divisor]
       CLR  *R6+           AND CLEAR IT OUT [clearing
       CLR  *R6+              8-byte
       CLR  *R6+                 floating point
       CLR  *R6                     workspace]
       CI   R0,0                                        **les**
       JEQ  CIF03          IF ZERO THEN DONE
*
       MOV  R0,R5          save sign                    **les**
       ABS  R0             ENSURE ITS POSITIVE
       LI   R3,>40         GET THE EXPONENT  [>40 is bias]
       C    R0,R7          < 100?
       JL   CIF02          yes; R100 exponent will be 0 (1 R100 digit)
       CI   R0,10000       < 10000?
       JL   CIF01          yes; R100 exponent will be 1 (2 R100 digits)
       INC  R3             no; inc R100 exponent (3 R100 digits)
       MOV  R0,R1          set up
       CLR  R0                for division
       DIV  R7,R0          divide int by 100 to get R100 rem
       SWPB R1             get 3rd significant R100 digit (rem) to MSB
       MOVB R1,@3(R4)      copy to 3rd significant R100 digit in FAC
CIF01  INC  R3             inc R100 exponent
       MOV  R0,R1          set up
       CLR  R0                for division
       DIV  R7,R0          divide remaining int by 100 to get R100 rem
       SWPB R1             get 2nd significant R100 digit (rem) to MSB
       MOVB R1,@2(R4)      copy to 2nd significant R100 digit in FAC
CIF02  SWPB R0             get 1st significant R100 digit to MSB
       MOVB R0,@1(R4)      copy to 1st significant R100 digit in FAC
       SWPB R3             get R100 exponent to MSB
       MOVB R3,*R4         copy to R100 exponent in FAC
       INV  R5             need to make FP number in FAC negative?
       JLT  CIF03          no; we're good
       NEG  *R4            yes; negate 1st word of FAC
CIF03  RT                  ; return to caller...

 

...lee

[Asmusr]

Yesterday I looked into implementing TIPI support in JS99er. My idea was to use websockets as the interface between JS99er/TI and the python code, which would be slow, but I don't think that would matter for the emulation. I got stuck because the python code is implemented as a polling loop instead of being event driven. What would it take to reverse control so that TipiMessage is notified when a byte has been received instead of polling for it? For the normal code branch I guess it would mean moving the polling loop into libtipi. 

[+jedimatt42]

3 minutes ago, Asmusr said:

Yesterday I looked into implementing TIPI support in JS99er. My idea was to use websockets as the interface between JS99er/TI and the python code, which would be slow, but I don't think that would matter for the emulation. I got stuck because the python code is implemented as a polling loop instead of being event driven. What would it take to reverse control so that TipiMessage is notified when a byte has been received instead of polling for it? For the normal code branch I guess it would mean moving the polling loop into libtipi. 

 

I wrote up how the messaging works here: https://github.com/jedimatt42/tipi/wiki/TIPI-Protocol

 

The hardware is basically a shared-memory approach... with some mailboxes, as you've discovered.  Both sides just check their mail... I imagined emulation would just completely replace libtipi.so with a shared memory implementation... If you don't interact with the 4A for every byte, you'd also have to change the ROM messaging code to match. The 4A side is all abstracted on top of the sendmsg, vsendmsg, recvmsg, and vrecvmsg routines.  You could tap in there on the 4A, replacing it to just stream the message over your websocket... I imagine your emulator can use magic memory that reads a byte from the socket or writes a byte to the socket.

 

and the python side could modify the TipiMessage abstraction to skip the libtipi byte-shiftregister level stuff, and the python can just read from that socket in the read message python routine... conversely for send message...    

[Asmusr]

There is no issue on the TI/emulator side. The problem in on the python side where I need to use something like https://websockets.readthedocs.io, which is event driven/asynchronous. It's not possible to poll a websocket, so it doesn't plug into TipiMessage as it is.

[+9640News]

I've got a development idea if there is someone out there wanting to tackle it for expanding the TIPI capabilities.  Matt has already laid out the ability to do something like PI.PIO.(variables) to control output to various settings, so someone probably has a decent starting point.

 

What I am suggesting is something like PI.JPG.GIF.FILE=DSK1.MUGSHOT or PI.JPG.MYART.FILE=DSK1.MUGSHOT, or PI.GIF.MYART.FILE=DSK1.MUGSHOT  that would take an existing file at a path within the TIPI directory, and convert it on the fly to a GIF or MYART file for the TI-99/4A and/or Geneve for file loading.  I've got assembly code that lays out the MyArt specification and decoding of those images.  I reference the MyArt format as it is very easy to decode and lay out on a 64K or 128K page for the 9938.

 

I've been looking out there and there are PYTHON scripts for JPEG to GIF, so just a thought if that is something someone would want to consider.

 

Just tossing it out there as something that could be very interesting as well with the 9938 or maybe (??) the F18a or the MK2.  I don't know what the limits are with the special features of the F18a or future MK2.

 

Just an idea if someone had an interest.

 

I know the Geneve has Picture Transfer that can display a GIF and save it as a MyArt picture which works well for a 9938 setup.  Not sure I have ever seen something for the TI-99/4A that could do that.  So having something that would display, and potentially save the file could be useful for various graphics, games, etc.

 

Just an idea...………...

 

Beery

 

 

 

[+jedimatt42]

3 minutes ago, BeeryMiller said:

I've got a development idea if there is someone out there wanting to tackle it for expanding the TIPI capabilities.  Matt has already laid out the ability to do something like PI.PIO.(variables) to control output to various settings, so someone probably has a decent starting point.

 

What I am suggesting is something like PI.JPG.GIF.FILE=DSK1.MUGSHOT or PI.JPG.MYART.FILE=DSK1.MUGSHOT, or PI.GIF.MYART.FILE=DSK1.MUGSHOT  that would take an existing file at a path within the TIPI directory, and convert it on the fly to a GIF or MYART file for the TI-99/4A and/or Geneve for file loading.  I've got assembly code that lays out the MyArt specification and decoding of those images.  I reference the MyArt format as it is very easy to decode and lay out on a 64K or 128K page for the 9938.

 

I've been looking out there and there are PYTHON scripts for JPEG to GIF, so just a thought if that is something someone would want to consider.

 

Just tossing it out there as something that could be very interesting as well with the 9938 or maybe (??) the F18a or the MK2.  I don't know what the limits are with the special features of the F18a or future MK2.

 

Just an idea if someone had an interest.

 

I know the Geneve has Picture Transfer that can display a GIF and save it as a MyArt picture which works well for a 9938 setup.  Not sure I have ever seen something for the TI-99/4A that could do that.  So having something that would display, and potentially save the file could be useful for various graphics, games, etc.

 

Just an idea...………...

 

Beery

 

 

 

 

Tursi's suggestion when TIPI prototype was first revealed was to provide something more natural... Like I do with XB source to PROGRAM conversion. 

 

If you try to LOAD opcode (or OPEN a DIS/FIX 128) a pc GIF or other image type, then the PI could trasform on the fly into a TIArtrist PROGRAM IMAGE file. 

 

I think you and InsaneMultitasker are the only ones running TIPIs on Geneves... so audience for Myart and GIF are pretty tiny... 

 

But, the same automatic mechanism could be modified to take target format hints... such as writing to PI.CONFIG a line like IMAGE_FORMAT=MYART, then try loading a JPG, and you get a Myart... 

 

That's the way I'd like to see that sort of interface work... convert in memory on load, leave native file in native file format on native filesystem. 

 

What do you want to do with these images ? How do you want to consume them?