Jump to content
Sign in to follow this  
retroclouds

Opinions on TI-99/4a text file formats

Recommended Posts

I'm currently implementing file access in my TiVi editor. The editor is designed to run in 80 columns mode.

 

My question is what text file formats should be supported? I mean DIS/VAR 80 for sure. But are there editors out there that support other file formats as well.

Does it make sense to implement scrolling in the editor so that I can let say open a file with 255 record length and show it on a single line with scrolling?

 

Any opinions on that?

 

 

 

Share this post


Link to post
Share on other sites

In my own work in the 80s, I used DF128 for text. It was a conversion from DV80 to MSDOS style. Packed into 128 byte records with CRLF line breaks. I went the other direction from txt files I downloaded. 

 

For TI-Net BBS I packed text into DF80 with line breaks so you could still have a limited chance of editing it. It loads much faster than DV80 in XB. 

 

BUt I don’t know any other apps that worked this way. 

 

Editing and saving DF80 I guess is the 2nd most common use case after DV80. The E/A editor works on object files. 

 

What about editing FORTH blocks? DF128 with 1K blocks of 16 lines by 64 chars. Opening this as lines split into equal 64 char lines would be neat. 

 

TI Artist instanced? Weren’t those a weird record length?

 

I think the ability to open any type of file with scrolling is a good idea but I can’t think of any other use cases. 

Share this post


Link to post
Share on other sites
4 hours ago, retroclouds said:

I'm currently implementing file access in my TiVi editor. The editor is designed to run in 80 columns mode.

 

My question is what text file formats should be supported? I mean DIS/VAR 80 for sure. But are there editors out there that support other file formats as well.

Does it make sense to implement scrolling in the editor so that I can let say open a file with 255 record length and show it on a single line with scrolling?

 

Any opinions on that?

 

Well, fbForth and TurboForth both use DIS/FIX 128 for their blocks files with no end-of-line markers. Their respective editors treat the files, artificially, as 1024-char blocks with 16 lines of 64 characters each. That is, each record contains exactly two pseudo lines and there is no division into blocks evident in the file itself. Also, even though the format is defined as “display”, blocks can contain non-ASCII characters, most notably as a result of BSAVE .

 

...lee

Share this post


Link to post
Share on other sites

CAMEL99 Forth also adopted that file convention for block files to maintain compatibility across the three systems. 

 

More relevant to this conversation I have been considering using the same system as a virtual memory manager for a text file editor.

This would mean on open you would transfer the text file, like DV80 for example, Into the "block" file and close the text file.

This would allow text files to be as large as the entire disk if need be. (on a two disk system)

 

The virtual memory system (block file) could have 4 to 7 large buffers in LOW RAM which make it respond like an editor with the text in RAM until you moved outside what was in the buffers. When that happened there is a momentary disk read/write but in a Forth editor it's only 1 second or 2 on a floppy system.

 

On SAVE the block file would be copied back to DV80 format removing any trailing spaces in the lines to conserve space.

 

Any thoughts by the group on this architecture? 

My one concern is what happens when you try and insert a line and it has to ripple down the entire virtual memory file.  That could be ugly.

 

Share this post


Link to post
Share on other sites
1 hour ago, TheBF said:

 

On SAVE the block file would be copied back to DV80 format removing any trailing spaces in the lines to conserve space.

 

Any thoughts by the group on this architecture? 

My one concern is what happens when you try and insert a line and it has to ripple down the entire virtual memory file.  That could be ugly.

 

Ugly.. what about moving lines by rearranging pointers? This is how TI Writer worked. There would be incoherence between the block file and the ram cache after any write but speed would be preserved. 

Share this post


Link to post
Share on other sites

All feedback so far from the forth guys! Excellent ;-)

DF80 seems like a candidate. Probably will go with some form of scrolling as well. Will need that when displaying 2 panes on the screen anyway.

 

 

 

 

 

Share this post


Link to post
Share on other sites
16 minutes ago, FarmerPotato said:

Ugly.. what about moving lines by rearranging pointers? This is how TI Writer worked. There would be incoherence between the block file and the ram cache after any write but speed would be preserved. 

That's basically how I do it in TiVi. It's copy-on-write and working with pointers. That allows me to quite easily add undo functionality.

Basically I edit a line and as soon as the line is "commited" by pressing enter (or moving to another line) the line is "packed/crunched" and added to the editor buffer.

What that means is, if I edit the same line multiple times (after leaving that line) I have multiple revisions in memory.

 

Still a bit worried about fragmentation but will see how that goes.

By "packing/crunching" I mean that the format how the text is kept in memory could be changed depending on the type of program I'm editing (e.g. BASIC).

For now it's basically a plain text copy with the actual line length.

Share this post


Link to post
Share on other sites

Yes that would be the solution, but it's a lot more complicated I think. Have not thought that through.

Here is how I understand it. (corrections please)

  • So in the virtual memory image the lines of text would be a linked list with a pointer at the head of each line?
  • Inserting a line would simple re-link a new line at the end of the file in between the two that were having the insertion place in between
  • Deleting a line simply re-links around the deleted line.
  • Displaying the file means traversing the linked list onto the screen. Not as fast but should not be too bad.
  • To write the file out to DV80 you traverse the list and write the entire file in list order.

 

Might not be too tricky.  If the pointer size is 16 bits the file size is limited to 64K bytes. Also acceptable IMHO.

 

Share this post


Link to post
Share on other sites

Like FarmerPotato, I use 'packed' DF128 files (msdos style) with the crlf page breaks for some applications. Fairly easy to display and works well when transferring between systems. The final record can be a pain without an end of file marker, such as null character or ctrl-Z  (ascii 26).   Ultimately what you select depends on the problem (or method) you are trying to solve for...

Share this post


Link to post
Share on other sites

I like the idea of using the cr/lf  with ^Z  file terminator, but I am torn with staying within the norm for text files for the TI-99.

 

Cheers for now.  I have to run.

 

 

Share this post


Link to post
Share on other sites

TheBf said

Quote

Might not be too tricky.  If the pointer size is 16 bits the file size is limited to 64K bytes. Also acceptable IMHO.

I imagined the pointers would only be for the in memory buffers. Not linked lists, just an array pointing to records. 

  • Like 1

Share this post


Link to post
Share on other sites
9 minutes ago, TheBF said:

Yes that would be the solution, but it's a lot more complicated I think. Have not thought that through.

Here is how I understand it. (corrections please)

  • So in the virtual memory image the lines of text would be a linked list with a pointer at the head of each line?
  • Inserting a line would simple re-link a new line at the end of the file in between the two that were having the insertion place in between
  • Deleting a line simply re-links around the deleted line.
  • Displaying the file means traversing the linked list onto the screen. Not as fast but should not be too bad.
  • To write the file out to DV80 you traverse the list and write the entire file in list order.

 

Might not be too tricky.  If the pointer size is 16 bits the file size is limited to 64K bytes. Also acceptable IMHO.

 

 

Basically the way I implemented is, that the lines of text are not a linked list with a pointer at the head. They are basically just concatenated text.

I keep the index with the pointers in a separate memory region, that makes it more easily for using SAMS (I guess, will have to see how that works out).

 

This is how the index looks like.

Each index slot entry 4 bytes each
*  Word 0: pointer to string (no length byte)
*  Word 1: MSB=Packed length, LSB=Unpacked length

 

Processing a range of lines is quite easy as I can easily calculate the offset in the index and go from there.

Deleting/Inserting lines is easy as well as I just rework the index. 


As said I will have to "defragment" the memory region holding the actual text once it runs full, but will work on that when I get there.

Dunno how big the slowdown will be, that might get interesting then. So far I'm pleased with the results. Working on a couple of hundred lines is very fast.

But now I first have to get file loading/saving in place so that I can get to work on bigger texts.

 

 

 

 

 

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...