TIPI - TI-99/4A to Raspberry PI interface development

[+jedimatt42]

26 minutes ago, Vorticon said:

I checked the log and there is nothing unusual in there. On the other hand, I switched out my FG for a real EA module and now the physical drives are accessible again, so it looks like TIPI does not like the EA II image I have on the FG. What was different with that EA version again?

 

aH... EA II ... so is it fixed in Fred's EA V r6 ? and we can ignore it  I admit I tested with an TI EA GROM.

Edited December 20, 2020 by jedimatt42

[+jedimatt42]

3 hours ago, Vorticon said:

I checked the log and there is nothing unusual in there. On the other hand, I switched out my FG for a real EA module and now the physical drives are accessible again, so it looks like TIPI does not like the EA II image I have on the FG. What was different with that EA version again?

 

Fred's EA II has his linking loader for option 3, the editor and assemblers in ROM, and he says he rewrote the loader, but it isn't clear from his page if that is just the EA3 linking loader, or the option 5 loader. 

 

For any contemporary software, I would just grab the latest and re-test. I believe his EA V has his debugger and improved editors as well built into the ROM.

[wolhess]

12 hours ago, Vorticon said:

Yes, but did you try using a real disk drive with an EA5 program on a diskette and loading it from the EA module directly?

Yes, I used three physical disk drives DSK1, DSK2 and DSK3 to load the EA5 program EDIT40 from the EA module!

 

Before I unmapped these drives from the tipi system, so I have direct access to the real floppy disks.

 

I also have the tipi on cru:1000 and the TI-Controller on cru:1100 and I used the TI-EA module from the FG99.

 

EDIT: I just tested with Fred'S EA IV and EA V (newest release) and it doesnt't load from option 5!

         Then I tested it with tursis EA complete and with this it is working fine like with the original EA module. 

[+Vorticon]

1 hour ago, wolhess said:

Yes, I used three physical disk drives DSK1, DSK2 and DSK3 to load the EA5 program EDIT40 from the EA module!

 

Before I unmapped these drives from the tipi system, so I have direct access to the real floppy disks.

 

I also have the tipi on cru:1000 and the TI-Controller on cru:1100 and I used the TI-EA module from the FG99.

 

EDIT: I just tested with Fred'S EA IV and EA V (newest release) and it doesnt't load from option 5!

         Then I tested it with tursis EA complete and with this it is working fine like with the original EA module. 

Thanks for checking!

Can you please post Tursi's EA complete cart image? It does not seem to be listed in the FG repository. Also what is different with this image as compared to the standard EA module?

[wolhess]

43 minutes ago, Vorticon said:

Can you please post Tursi's EA complete cart image? It does not seem to be listed in the FG repository. Also what is different with this image as compared to the standard EA module?

Hi,

The difference is that in tursis EA module the editor and asembler are in the module and do not require a disk.

 

Here is the bin: NEWEAG.BIN

[+Schmitzi]

 

Hi,

 

is this possible maybe ?

(HighScore)

 

 

 

[+9640News]

Matt,

 

Testing here is in Geneve Rompage mode, so no Geneve DSR in play.

 

Been trying to chase down a catalog issue I having with the TIPI in MDOS/Assembly, so I put together an extended basic program for testing.  I am including a copy of the extended basic program I am using below.  When I run the program, with my directory listing I get an error after the third line of display.  

 

With the Myarc HFDC, if I run the same program substituting WDS1 for TIPI in line 210, I get a listing of all files, then it continues on and I then get a list of directories.

 

If the TIPI was presenting a mixture by alphabetical order, then I think I can deal with that in my other coding efforts.  However with the error being generated here in extended basic, I think there may be an issue???

 

With the TIPI, if I run the extended basic program, I get everything in alphabetical order until it hits the first file after a directory is displayed, then I get an error.

 

I think whatever is happening here is what is confusing my assembly code.  Since it does generate an error in extended basic, I wanted to report it.

 

 

 

CATALOG

Edited December 22, 2020 by BeeryMiller

[+jedimatt42]

3 hours ago, BeeryMiller said:

Matt,

 

Testing here is in Geneve Rompage mode, so no Geneve DSR in play.

 

Been trying to chase down a catalog issue I having with the TIPI in MDOS/Assembly, so I put together an extended basic program for testing.  I am including a copy of the extended basic program I am using below.  When I run the program, with my directory listing I get an error after the third line of display.  

 

With the Myarc HFDC, if I run the same program substituting WDS1 for TIPI in line 210, I get a listing of all files, then it continues on and I then get a list of directories.

 

If the TIPI was presenting a mixture by alphabetical order, then I think I can deal with that in my other coding efforts.  However with the error being generated here in extended basic, I think there may be an issue???

 

With the TIPI, if I run the extended basic program, I get everything in alphabetical order until it hits the first file after a directory is displayed, then I get an error.

 

I think whatever is happening here is what is confusing my assembly code.  Since it does generate an error in extended basic, I wanted to report it.

 

 

 

CATALOG 720 B · 3 downloads

 

Was this normal that directories were only listed at the end by _all_ devices that support directories? ( I agree it should be irrelevant. ) 

 

[+jedimatt42]

11 minutes ago, jedimatt42 said:

 

Was this normal that directories were only listed at the end by _all_ devices that support directories? ( I agree it should be irrelevant. ) 

 

@Beery, IDK, your catalog program errors for me after the first file... What is the expected value of K on line 300?

 

[+9640News]

3 hours ago, jedimatt42 said:

@Beery, IDK, your catalog program errors for me after the first file... What is the expected value of K on line 300?

 

The above program was typed in from the TI Disk Controller manual.  I pulled the controller manual copy up from the CYC/ page 48.  

 

On the Myarc HFDC, the above program does not error out.  However, on the TIPI and SCS1, it will error out. 

 

I just looked through program listing, and in line 300, B$="<space><space>" should have been B$="<space><space><space>".  this avoids the error being generated in line 310 with a neggative SEG$.

 

On the Myarc HFDC and the SCSI, the above program gives files first, then a list of directories, each set in alphabetical order.  TIPI provides the listing of all files/dirs in alphabetical order.

 

As far as what is the value of K, I do not know.

 

Beery

 

 

 

[+jedimatt42]

My real system is down for the count right now... The SAMS card failed yesterday - hopefully the same issue can be reproduced under the emulation... but so far, files after directories is not a problem... 

[+jedimatt42]

Ah, K is just the max record length for record based file types.

 

Hmm... This just still works for me... Made the adjustment to line 300, which looks like padding for right justification of the record length.

 

100 CALL CLEAR
110 DIM TYPE$(6)
120 TYPE$(1)="DIS/FIX"
130 TYPE$(2)="DIS/VAR"
140 TYPE$(3)="INT/FIX"
150 TYPE$(4)="INT/VAR"
160 TYPE$(5)="PROGRAM"
165 TYPE$(6)="<DIR>  "
210 OPEN #1:"TIPI.",INPUT ,RELATIVE,INTERNAL
220 INPUT #1:A$,J,J,K
230 DISPLAY "TIP1";" - DISKNAME = ";A$:"AVAILABLE = ";K;"USED = ";J-K
240 DISPLAY :"FILENAME    SIZE  TYPE    P":"---------------------------"
250 FOR LOOP=1 TO 127
260 INPUT #1:A$,A,J,K
270 IF LEN(A$)=0 THEN 350
280 DISPLAY :A$;TAB(12);J;TAB(17);TYPE$(ABS(A));
290 IF ABS(A)=6 THEN 320
300 B$="  "&STR$(K)
310 DISPLAY SEG$(B$,LEN(B$)-2,3);
320 IF A>0 THEN 340
330 DISPLAY TAB(28);"Y";
340 NEXT LOOP
350 CLOSE #1

 

Maybe it is specific to the file that you have following the directory... if you ssh to linux, and go into the tipi_disk folder, and then ls -l you should see them in the same order TIPI will present. 

[+9640News]

2 hours ago, jedimatt42 said:

My real system is down for the count right now... The SAMS card failed yesterday - hopefully the same issue can be reproduced under the emulation... but so far, files after directories is not a problem... 

Wasn't quite sure if the end of that last statement was an indication you were going to make that modification, or it is in reference to the extended basic catalog program.

 

If it is not much of an effort, presenting a catalog listing with directories before files like that would be greatly appreciated. If on the other hand, some kind of string can be written back out to PI.CONFIG(?) that would change that behavior, that would work as well.

 

Beery

 

[+jedimatt42]

2 minutes ago, BeeryMiller said:

Wasn't quite sure if the end of that last statement was an indication you were going to make that modification, or it is in reference to the extended basic catalog program.

 

If it is not much of an effort, presenting a catalog listing with directories before files like that would be greatly appreciated. If on the other hand, some kind of string can be written back out to PI.CONFIG(?) that would change that behavior, that would work as well.

 

Beery

 

I was just saying, I don't see an error... I think you indicated that encountering a file after encountering a directory causes an error... but I list things fine in a mixed type order... 

 

I am open to splitting it.. If that was the convention. Making it something that can be turned off appeals to me.

 

But we have to clarify... 

 

If files come after directories, you get an error with legacy software...? 

Legacy controllers HFDC and SCSI list directories after all the files? 

But you'd like the directories listed first, before the files?

 

[+9640News]

4 minutes ago, jedimatt42 said:

I was just saying, I don't see an error... I think you indicated that encountering a file after encountering a directory causes an error... but I list things fine in a mixed type order... 

 

I am open to splitting it.. If that was the convention. Making it something that can be turned off appeals to me.

 

But we have to clarify... 

 

If files come after directories, you get an error with legacy software...? 

Legacy controllers HFDC and SCSI list directories after all the files? 

But you'd like the directories listed first, before the files?

 

Well, to stay consistent with the TI-99/4A, cataloging with the Extended Basic program, files come before directories with the SCSI and Myarc HFDC controller cards.  That would be how those DSR's present the catalog listing. 

 

For that other system I use, directories are presented first.  I don't know if there is a setting that could be set in PI.CONFIG (???) that could give the user configuration control like files first, directories first, or alphabetical listing to control the user presentation of the catalog.

 

When I run DM2K on the Myarc HFDC under a TI configuration, files are presented first, then directories. With the TIPI and DM2K, it appears to be the loading order is strictly alphabetical so it can handle either way so it would not break that software.  In that respect DM2K is very versatile and covers both scenarios.

 

Off the top of my head, and outside of your program force command and DM2K, I am not sure what other TI-99/4A programs are out there that can do a directory on TIPI with file operations that also contain subdirectories.  Really old legacy software that knew nothing about directories would likely default a directory  to one of the other filetypes dependent upon what the program defaulted an unknown filetype.

 

My extended basic program I posted earlier, added the TYPE$(6) = "<DIR>   " as I knew it.  That wasn't thought about during the early TI days.

 

Beery

 

 

[wolhess]

On 12/22/2020 at 4:14 PM, BeeryMiller said:

With the TIPI, if I run the extended basic program, I get everything in alphabetical order until it hits the first file after a directory is displayed, then I get an error.

Hi Beery,

 

I tried my XB catalog program on real TI with TIPI and I can show all 3000+ files of the DL folder. It takes more than 30 minutes to display all files.

  

 

 

The XB program shows the files all in alphabethical order and if I have folders there are also in alphabetical order between or with normal files.

 

 

 

Here is my version of the XB CAT-program modified but based on the one from the TI Disk-manual:

It accept 1-9 and A-Z for DSK1. - DSK9. and DSKA. - DSKZ.

It also accept a TIPI path like TIPI.AH.DL with or without the last dot.

The program maps a tipi path to "DSK4." and if you only have entered "TIPI" or "TIPI." then "DSK0." is used.

 

XDIR

 

 

[+9640News]

Matt,

 

Not sure if this is an issue on the PI end or not....

 

I tried writing a file with the first character in the filename containing the hex byte >FF.  I have this information in the TIPI log where it only wants characters below ASCI 128.  No issue there, as I understand that.  Is the TIPI code not passing some kind of error code response back in this case when it processes an unhandled exception? It looks like my code is waiting for a response (good or bad), and it isn't getting any response back.

 

The hex byte >FF was supposed to be a flag to indicate to skip the file and not write it, so I am chasing that part down myself. 

 

Beery

 

2020-12-24 12:16:59,034 tinames.tinames: INFO     TIPI.DOWN2.AAA. -> /home/tipi/tipi_disk/DOWN2/AAA
2020-12-24 12:16:59,034 LevelTwo    : INFO     set unit 0 path to TIPI.DOWN2.AAA.
2020-12-24 12:16:59,035 LevelTwo    : INFO     direct output
2020-12-24 12:16:59,038 TipiService : ERROR    Unhandled exception in main
Traceback (most recent call last):
  File "./TipiService.py", line 65, in <module>
    if levelTwo.handle(msg):
  File "/home/tipi/tipi/services/LevelTwo.py", line 36, in handle
    return handler()
  File "/home/tipi/tipi/services/LevelTwo.py", line 231, in handleDirectOutput
    filename = str(self.tipi_io.receive(), 'ascii').strip()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
2020-12-24 12:16:59,480 TipiService : INFO     physical mode enabled
2020-12-24 12:16:59,481 TipiService : INFO     TIPI Ready
 

 

 

[+jedimatt42]

1 hour ago, BeeryMiller said:

Matt,

 

Not sure if this is an issue on the PI end or not....

 

I tried writing a file with the first character in the filename containing the hex byte >FF.  I have this information in the TIPI log where it only wants characters below ASCI 128.  No issue there, as I understand that.  Is the TIPI code not passing some kind of error code response back in this case when it processes an unhandled exception? It looks like my code is waiting for a response (good or bad), and it isn't getting any response back.

 

The hex byte >FF was supposed to be a flag to indicate to skip the file and not write it, so I am chasing that part down myself. 

 

Beery

 

2020-12-24 12:16:59,034 tinames.tinames: INFO     TIPI.DOWN2.AAA. -> /home/tipi/tipi_disk/DOWN2/AAA
2020-12-24 12:16:59,034 LevelTwo    : INFO     set unit 0 path to TIPI.DOWN2.AAA.
2020-12-24 12:16:59,035 LevelTwo    : INFO     direct output
2020-12-24 12:16:59,038 TipiService : ERROR    Unhandled exception in main
Traceback (most recent call last):
  File "./TipiService.py", line 65, in <module>
    if levelTwo.handle(msg):
  File "/home/tipi/tipi/services/LevelTwo.py", line 36, in handle
    return handler()
  File "/home/tipi/tipi/services/LevelTwo.py", line 231, in handleDirectOutput
    filename = str(self.tipi_io.receive(), 'ascii').strip()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
2020-12-24 12:16:59,480 TipiService : INFO     physical mode enabled
2020-12-24 12:16:59,481 TipiService : INFO     TIPI Ready
 

 

 

 

TIPI uses a linux filesystem in UTF-8 encoding mode for filenames. In linux, I don't think there is such a thing as an illegal character for a filename. Some are just really hard to work with in a shell.

 

99% of the places in TIPI source that say 'ascii' are wrong, and I should fix this. They should be 'latin-1' so that 0-255 are all legal values. That is the actual intent.  So this is an actual bug in TIPI from the python2 to python3 conversion effort.

 

(My TI is working again, so once I get done with prep for tomorrow, I can work on some of these TIPI bugs/requests)

 

When I fix this, writing an 0xFF as the first character is going to succeed to create a file with that as the first character. This sounds like a Myarc alternative to the un-implemented TI temporary file mechanism... is that true? TIPI does not implement either mechanism. So, if this is a request, is there more detail? I've never heard of this behavior before... maybe it was supposed to be handled entirely in the Geneve OS master DSR?

 

As for unhandled exceptions, yes they are unhandled... handled exceptions return an error to the TI.

 

Unhandled exceptions just crash, and cause you to have to reset the 4A, so that you get annoyed and report the issue. LOL.. At least that part seems to be working  

 

 

[jrhodes]

1 hour ago, jedimatt42 said:

 

I don't think there is such a thing as an illegal character for a filename.

 

 

Under Linux and other Unix-related systems, there are only two characters that cannot appear in the name of a file or directory, and those are NUL '\0' and slash '/' .

Go ahead, try and create a linux file named / , bet you can't do it.

You also can not create files named "." or ".."

Edited December 24, 2020 by jrhodes

[+9640News]

22 minutes ago, jedimatt42 said:

When I fix this, writing an 0xFF as the first character is going to succeed to create a file with that as the first character. This sounds like a Myarc alternative to the un-implemented TI temporary file mechanism... is that true? TIPI does not implement either mechanism. So, if this is a request, is there more detail? I've never heard of this behavior before... maybe it was supposed to be handled entirely in the Geneve OS master DSR?

 

There is no request to support the 0xFF filenaming, rather reporting the bug as you described.  Don't recall an alternative to the un-implemented TI temporary file mechanism you mentioned, and not something I am seeking/requesting.  

 

Just some background on what I am doing as I did not know your timetable or what configuration option(s) you were going to implement.

 

There are two commands in the command line interpreter in MDOS that pull the same routine to catalog a path  They are the COPY command, and the DIR command.  You are allowed to do a wildcard search for a name.

 

When this GETCATS routine is called, it places a list of up to 114 directory names and 127 filenames into two different buffers that meet the search criteria.  So, if you search for "AA*" or "*", you will only get directories and files populated into those two separate lists that meet the criteria.

 

With the HFDC, everything initially comes in as subdirectories first as the master dsr is written, then when the code no longer detects any subdirectories, it switches to populating the files list.  Each name as it is pulled is individually searched during the read to see if it matches the search criteria.

 

Now, after the DIR call gets the device cataloged, then it displays the directories first, then it goes on to the list of files that meet the criteria.

 

The GETCATS code that builds these two lists is not smart enough to split files and directories if they come in mixed.  So, register usage goes afoul trying to deal with both simultaneously.  The code as originally written never envisioned such a scenario which makes it a nightmare to manage the registers and pointers.  I tried some code to handle switching back and forth, but there were still pointers somewhere that were being reset.  So, what I did was treat the list of everything coming in from the TIPI during the cataloging process as a filename.  Mixed in with all that code was some other code that also creates a fake file control block in memory to quicken the display output processing when a DIR is called.  After a few code tweaks, and directories in that list are properly display as a directory listing when a TIPI is cataloged in a mixed file listing as it is currently seen.

 

As the COPY routine is only concerned about a path whether it is doing a complete directory, or a "AA*" (ex.), it would pick up a directory name AAA if it existed that would go into the FILES list on the TIPI device.  In that fake file control block is a designator that it is a DIR and not a D/F, D/F, I/F, etc file format.

 

So, now the COPY command reads the FILES list, and goes to buffer as many of the files it can in memory dependent upon available memory.  There are a host of routines depending upon the type of file, and to whether it is being written out to floppy disk, rs232, pio, hard disk, etc.

 

As the code starts reading through the list of FILE names that are marked as a DIR in the fake file control block, I mark the  filename (that is actually a subdirectory) with the first character as >FF so the code will know to avoid the call to the OPEN Pab call and jump over it.  Somewhere, there is call I have not isolated that still looked to copy that subdirectory (as file) even though I thought I had code that would have jumped over it. Because it had a filename beginning with >FF, it generated that error.  Once I find it, the error should not be generated.

 

If the requests you mentioned have the option of grouping directories,  then there will only need to be very minor code changes. As it is now, the code works fine if I tell it to do a COPY AA* and there are no directory names that meet that criteria as well.  It is only when I test across the use of the COPY command with directories that get also added to the list.

 

Anyways, that is probably more than you wanted to know.  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

[+9640News]

Matt,

Wasn't sure which topic area to put this, but came across another error at a "safepoint".  Log section below.  Basically, I went to copy a bunch of files from a floppy drive to a TIPI sub folder with a COPY * command.  All files were buffered before the code then tries to start writing them out.  The first filename in the list is $MAKE.  I've copied that filename multiple times in the past with no issue. Things locked up on the PI.  I rebooted the PI, and repeated the procedure, and then things copied fine so I don't know at this point how to duplicate the issue.  Just giving you some feedback if you see something there.

 

BTW, found the issue I was trying to chase down above.  Let's just say it was buried in some sneaky code.

 

2020-12-24 19:25:39,764 LevelTwo    : INFO     set path request
2020-12-24 19:25:39,767 LevelTwo    : INFO     unit: 0, path: TIPI.DOWN2.AAA.
2020-12-24 19:25:39,767 tinames.tinames: INFO     TIPI.DOWN2.AAA. -> /home/tipi/tipi_disk/DOWN2/AAA
2020-12-24 19:25:39,768 tinames.tinames: INFO     TIPI.DOWN2.AAA. -> /home/tipi/tipi_disk/DOWN2/AAA
2020-12-24 19:25:39,768 LevelTwo    : INFO     set unit 0 path to TIPI.DOWN2.AAA.
2020-12-24 19:25:39,769 LevelTwo    : INFO     direct output
2020-12-24 19:25:40,428 TipiService : ERROR    Unhandled exception in main
Traceback (most recent call last):
  File "/home/tipi/tipi/services/tipi/TipiMessage.py", line 22, in receive
    message = self.ports.readMsg()
  File "/home/tipi/tipi/services/tipi/TipiPorts.py", line 21, in readMsg
    return tipiports.readMsg()
ValueError: safepoint

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./TipiService.py", line 65, in <module>
    if levelTwo.handle(msg):
  File "/home/tipi/tipi/services/LevelTwo.py", line 36, in handle
    return handler()
  File "/home/tipi/tipi/services/LevelTwo.py", line 232, in handleDirectOutput
    bytes = self.tipi_io.receive()
  File "/home/tipi/tipi/services/tipi/TipiMessage.py", line 24, in receive
    raise BackOffException('safepoint')
tipi.TipiMessage.BackOffException: safepoint
2020-12-24 19:25:40,944 TipiService : INFO     physical mode enabled
2020-12-24 19:25:40,945 TipiService : INFO     TIPI Ready
 

 

 

 

[+jedimatt42]

5 hours ago, jrhodes said:

Go ahead, try and create a linux file named / , bet you can't do it.

I love you too!

[+jedimatt42]

5 hours ago, BeeryMiller said:

Just some background on what I am doing as I did not know your timetable or what configuration option(s) you were going to implement.

 

I plan to offer a DIRSORT option in PI.CONFIG with 3 values, MIXED, FIRST, LAST

 

I'll set the default to FIRST, as that sounds most usable, and what we see so often on other platforms, and sounds like it matches the HFDC.

 

I plan to work on it some tonight, might not take long, and then might be able to release an update Saturday morning. 

Edited December 25, 2020 by jedimatt42

[+9640News]

7 minutes ago, jedimatt42 said:

 

I plan to offer a DIRSORT option in PI.CONFIG with 3 values, MIXED, FIRST, LAST

 

I'll set the default to FIRST, as that sounds most usable, and what we see so often on other platforms, and sounds like it matches the HFDC.

 

I plan to work on it some tonight, might not take long, and then might be able to release an update Saturday morning. 

Love the plan!

[+jedimatt42]

There was still quite a bit if 'ascii' encoding of filenames in the python, that should have been latin1... we are just so conventional, that we aren't tripping on it. Most LVL3 IO would be happy with high characters, but none of the LVL2 routines would be... so you could create files in BASIC, that you can't COPY with a contemporary file manager. GOOD FIND!  (thanks @BeeryMiller ) And another half dozen found just through code searching.... 

Edited December 25, 2020 by jedimatt42