Large DIS/VAR 80 files ?

[+retroclouds]

I'd like to do a little experiment using the TiVi edtor I'm currently working on. 

What is the largest DIS/VAR 80 file you were able to load and browse on the TI-99/4a?

 

I'm trying to get my hands on some big DIS/VAR 80 files and with "big" I mean that it should be at least > 100 kilobytes.

If you have such file and don't mind posting it (as a disk image). I'll try to load it on the TI-99/4a.

 

For further details on the editor see here:

 

 

[+TheBF]

Give me a day. I should be able to make a script to combine a pile of source code files into one big file.

[+mizapf]

Here's one from me: MAME source code of the TMS9900 and the TMS9995. Never loaded them on the real iron, of course.

longdv80.dsk

[Tursi]

This took longer than planned, because in creating the disk image with Classic99, grumble, I found it didn't handle full disks very well.

 

Fixed now. So here is the majesty of my very own MLP FanFic, formatted for DV80 and presented on a DSDD floppy disk image. 1400 sectors even. 306k of text. Even TI99Dir won't read the whole thing

 

bigwork.dsk

 

[+retroclouds]

Thanks guys, really appreciate it a lot.

 

The largest file I loaded so far was Stuart Conner’s TI Invaders source code. That’s some >3000 lines of assembly source code.

With the files you all provide I’ll. have some bigger files to experiment with.

 

What I’ve learned so far is that I’ll need to speed up file loading, because using normal file I/O takes a long time even from ramdisk or TIPI. Also need to fine tune index handling. But I’ll take that discussion to my TiVi thread.

 

I’ll see if I can do a couple of videos on the TI-99/4a (no emulation).

[atrax27407]

With 80 column F'WEB, you can load approximately 4500 lines of source code. Of course, you need an AVPC or EVPC to do it.

[+retroclouds]

As promised here's a rather long video of me demoing loading large files into SAMS. 

 

 

[+TheBF]

21 hours ago, TheBF said:

Give me a day. I should be able to make a script to combine a pile of source code files into one big file.

Well... I had never tested my file util shell using APPEND mode and of course there was a bug. ?

But you have been well looked after and I can now APPEND files to my hearts content.

Thanks for forcing me to test better.   

 

I love the editor BTW.

 

 

[+TheBF]

I don't have any fancy drives on my TI-99 but even when I test loading files into SAMS memory on Classic99 it is pretty slow.

From what I can see on your video you are loading about 1.4K bytes per second.

 

On my SAMS editor on Classic99 I loaded a 13.6K file (the editor source) in about 7.5 seconds. That included erasing the 64K segment which took just over 1 second.

So just a touch faster but of course Classic99 may well be serving up the files much faster than TIPI. 

 

[RXB]

3 hours ago, TheBF said:

I don't have any fancy drives on my TI-99 but even when I test loading files into SAMS memory on Classic99 it is pretty slow.

From what I can see on your video you are loading about 1.4K bytes per second.

 

On my SAMS editor on Classic99 I loaded a 13.6K file (the editor source) in about 7.5 seconds. That included erasing the 64K segment which took just over 1 second.

So just a touch faster but of course Classic99 may well be serving up the files much faster than TIPI. 

 

Try using the RXB IN THE DARK game I am told it loads pretty fast off of disk for 584K.

[+TheBF]

27 minutes ago, RXB said:

Try using the RXB IN THE DARK game I am told it loads pretty fast off of disk for 584K.

Is it loading text files or program files?

The 99 goes like blazes on program loading.

[+retroclouds]

5 hours ago, TheBF said:

Is it loading text files or program files?

The 99 goes like blazes on program loading.

 

It is loading text files. There still a couple of tricks I can use to speed it up a bit. Probably can make better use of scratchpad RAM and for HDR I can copy directly to memory instead of hopping via VDP memory.

 

Do have the impression that the HDR ramdisk is a bit faster as TIPI. 

 

Thinking about it I’m going to keep the DIS/VAR 80 as an “import/export” functionality (for compatibility reasons with other software on the TI-99/4a). But also allow native saving/loading as program. 

[+TheBF]

3 hours ago, retroclouds said:

Thinking about it I’m going to keep the DIS/VAR 80 as an “import/export” functionality (for compatibility reasons with other software on the TI-99/4a). But also allow native saving/loading as program. 

I was thinking the same thing.  You could read/write the files in 4K chunks the way Rich is doing it. (i believe)

[RXB]

13 hours ago, TheBF said:

Is it loading text files or program files?

The 99 goes like blazes on program loading.

CALL BLOAD uses PROGRAM FILE Format it loads faster then any other type of file access.

RXB 2001 to RXB 2015 used 8K Program File Format but RXB 2020 will use 4K as that is best for SAMS 4K pages.

 

Edited May 3, 2020 by RXB
comment added

[Tursi]

On 5/3/2020 at 12:15 AM, retroclouds said:

 

It is loading text files. There still a couple of tricks I can use to speed it up a bit. Probably can make better use of scratchpad RAM and for HDR I can copy directly to memory instead of hopping via VDP memory.

 

Do have the impression that the HDR ramdisk is a bit faster as TIPI. 

 

Thinking about it I’m going to keep the DIS/VAR 80 as an “import/export” functionality (for compatibility reasons with other software on the TI-99/4a). But also allow native saving/loading as program. 

If you use the file sector access mechanism, you should see a fairly large speed up and most systems support it (not disk sector access, there's a way to read a file sector by sector). The advantage is fewer calls to the DSR - for each call you'll get a 256 byte sector which contains 3 or more D/V80 records (depending on how big they are). You have to parse them out yourself, but it's pretty straight forward.

 

[+InsaneMultitasker]

On 5/3/2020 at 2:15 AM, retroclouds said:

 

It is loading text files. There still a couple of tricks I can use to speed it up a bit. Probably can make better use of scratchpad RAM and for HDR I can copy directly to memory instead of hopping via VDP memory.

 

I've been toying with the idea of extending the current ROS direct input/output routines to allow for passing a SAMS pagelist pointer to the DSR, so that the DSR could load/save a file directly to/from RAM via a single operation. This assumes the file size is <= to the available memory and that the calling program properly constructs the page list.  If you get to the point where you are using >14 and >15 to load/save the large files and want to try a 'proof of concept'  with ROS and SAMS, let me know.

[+retroclouds]

12 hours ago, InsaneMultitasker said:

I've been toying with the idea of extending the current ROS direct input/output routines to allow for passing a SAMS pagelist pointer to the DSR, so that the DSR could load/save a file directly to/from RAM via a single operation. This assumes the file size is <= to the available memory and that the calling program properly constructs the page list.  If you get to the point where you are using >14 and >15 to load/save the large files and want to try a 'proof of concept'  with ROS and SAMS, let me know.

 

That’s an iteresting idea, I like it a lot. I’ll look into it and let you know.

 

One thing that comes to mind is, that it would be cool if during the file I/O the DSR would allow user defined functions to be called.

That way you could for example pass 2 pointers to the DSR:

1. Address of a function that provides the next target SAMS page
2. Address of a function that acts like a processing trigger each time after a SAMS page is loaded.

Reason for (2) would be that I store the file internally in memory not in the same format as it is on disk or could do steps to halt processing on some user condition.

 

Either way, would perhaps be sufficient to start with what you’ve proposed, passing a SAMS pagelist pointer.

 

[RXB]

12 minutes ago, retroclouds said:

 

That’s an iteresting idea, I like it a lot. I’ll look into it and let you know.

 

One thing that comes to mind is, that it would be cool if during the file I/O the DSR would allow user defined functions to be called.

That way you could for example pass 2 pointers to the DSR:

1. Address of a function that provides the next target SAMS page
2. Address of a function that acts like a processing trigger each time after a SAMS page is loaded.

Reason for (2) would be that I store the file internally in memory not in the same format as it is on disk or could do steps to halt processing on some user condition.

 

Either way, would perhaps be sufficient to start with what you’ve proposed, passing a SAMS pagelist pointer.

 

You mean like RXB does in the SAMS DEMO:

100 FOR P=1 TO 57
110 CALL SAMS(2,P+15) ! 2=>2000
120 CALL PLOAD(2,"DSK1.PAGE"&STR$(P))
130 CALL MOVES("RV",2079,8192,0)
140 NEXT P
160 FOR P=16 TO 57
170 CALL SAMS(2,P+15)
180 CALL MOVES("RV",2079,8192,0)
190 NEXT P
200 GOTO 160

Memory boundary 2=>2000, 3=>3000, A=>A000 and so on up to F=>F000

The above RXB SAMS DEMO loads 57 pages of 4K SAMS for a total of 4*57=228K of RAM SAMS memory.

Then it moves it to screen to show it.

 

[+InsaneMultitasker]

6 hours ago, retroclouds said:

 

That’s an iteresting idea, I like it a lot. I’ll look into it and let you know.

 

One thing that comes to mind is, that it would be cool if during the file I/O the DSR would allow user defined functions to be called.

That way you could for example pass 2 pointers to the DSR:

1. Address of a function that provides the next target SAMS page
2. Address of a function that acts like a processing trigger each time after a SAMS page is loaded.

Reason for (2) would be that I store the file internally in memory not in the same format as it is on disk or could do steps to halt processing on some user condition.

 

Either way, would perhaps be sufficient to start with what you’ve proposed, passing a SAMS pagelist pointer.

 

I don't quite follow the user defined function examples/ideas, can you elaborate?     I don't have much free time right now so I've added your information into my logs for future review.  It would be nice if a specific page of SAMS was reserved for memory page lists and support routines for paging.  Did anyone ever produce memory handling routines for SAMS?

[+retroclouds]

11 hours ago, InsaneMultitasker said:

I don't quite follow the user defined function examples/ideas, can you elaborate?     I don't have much free time right now so I've added your information into my logs for future review.  It would be nice if a specific page of SAMS was reserved for memory page lists and support routines for paging.  Did anyone ever produce memory handling routines for SAMS?

 

I’ll see if I can come with a good example in the next few days. But for now:

 

Callback “get next free SAMS page”

 

Basically the idea is that as a user you supply a pointer to a callback function that gets called again and again by the DSR while it’s loading the file.

Obviously the callback function should follow some conventions so that it’s safe to switch from the DSR to the callback and return to the DSR without crashing it.

 

One such example could be that instead of supplying the page list in advance, callback function would just supply the next “free” page each time the DSR requires one while loading the file.

 

 

Callback “SAMS page loaded”

 

Same idea, but at a later “trigger” point. As soon as a single SAMS page is loaded by the DSR this callback could be executed when set.

The callback could be used for doing something with the SAMS page (in my case that would be change internal memory structures because in memory the layout of my editor is not the same as written on disk).

Arguably these are things that can also be done after the file has fully loaded, but I think would offer more flexibility.

Another example could be doing a CRC check on the loaded SAMS page, so you can abort file loading if it doesn’t match.

 

 

Calllback “SAMS page saved”

 

Same idea as for “SAMS page loaded”. Perhaps this could be combined with the above one. Would need to think about that one a bit more ?

[+retroclouds]

On 5/2/2020 at 2:32 AM, mizapf said:

Here's one from me: MAME source code of the TMS9900 and the TMS9995. Never loaded them on the real iron, of course.

longdv80.dsk 360 kB · 11 downloads

 

Was able to successfully load the TMS9900 source code on the TI-99/4a. Thanks! ?