Text Files on the TI - Getting Past Size Limitations

[Omega-TI]

Excuse me if this has already been done, or talked about on the TI, but I've not noticed.

I apologize in advance for the long drawn out explanation, but not being a programming guru, I don't know all the correct words to make a concise explanation. 

 

As I currently understand it...

RAM is the limitation of how large a file can be on the TI, for editing, at one time. 

 

However, BITD...

I do not remember having this limitation on my TRS-80 Model III.  I believe it would load 'segments' of larger sized files to use and edit, and save it in some sort of appended format.  I do not know if that format had something to do with the disk format or not, but I'm assuming this to be the case.

 

Which brings me to wonder...

The FIAD format files we use today are independent of any disk file format, so even if a program could keep track of appropriate file header data, how would one 'edit segments' for re-integration into the proper location with the memory limitation that currently exists?

 

It would be cheating and not very TI...

I imagine some sort of TIPI/RPi integration with a TI program could accomplish the feat, however, for the record, I'm not even remotely suggesting this route.    

 

In the end...

It appears to me a whole new text file format would have to be developed, which is probably not going to happen due to how few would actually use such a program, even if it could be developed.  But from a strictly utilitarian purpose, it would be awesome to be able to write, import and maintain large documents on the TI without worrying about hitting a size limit.

 

A discussion on the feasibility and possible methods would be fascinating though...

[+retroclouds]

The internal representation of a text file in RAM is not necessarily the same (and for most text TI-editors it probably isn't, not sure there).

The programming editor I'm working on (at least from time to time) has a different representation as it has when written to disk.

My goal is to support large files (when using SAMS).

 

You can read about it in the TiVi thread.

 

[Tursi]

Technically there's nothing that makes a TI text file any harder to page than any other computer. It's just easier to write an editor that keeps everything in memory than one that pages.

 

Variable length records (DV80) would be at least slightly tricky to work with at the file level, as the DSR needs to enumerate the whole file to get to a specific record. You could work on it at the sector level, though, since a line is never split across sectors.

 

Fixed length records (DF80) is all but made for this, with instant seek to any record index possible.

 

[+Ksarul]

ISTR that Amnion's Wortex used sector-length records (I'll have to check my manual to be sure) to be able to do this--but Wortex was mostly in Extended BASIC and very slow, so it didn't ever get much of a user base.

[Omega-TI]

21 hours ago, Tursi said:

You could work on it at the sector level, though, since a line is never split across sectors.

Which brings me back to the original question, as an individual FIAD file, not in a disk format, say like on a TIPI SD card, would it even be possible?

[Tursi]

FIAD can be accessed at the sector level just like disk files -- one of the access modes will give you the file data a sector at at time. It's not the same as reading disk sectors.

 

I didn't miss your question criteria.

 

[+TheBF]

On 8/18/2019 at 1:49 PM, --- Ω --- said:

Excuse me if this has already been done, or talked about on the TI, but I've not noticed.

I apologize in advance for the long drawn out explanation, but not being a programming guru, I don't know all the correct words to make a concise explanation. 

 

As I currently understand it...

RAM is the limitation of how large a file can be on the TI, for editing, at one time. 

 

However, BITD...

I do not remember having this limitation on my TRS-80 Model III.  I believe it would load 'segments' of larger sized files to use and edit, and save it in some sort of appended format.  I do not know if that format had something to do with the disk format or not, but I'm assuming this to be the case.

 

Which brings me to wonder...

The FIAD format files we use today are independent of any disk file format, so even if a program could keep track of appropriate file header data, how would one 'edit segments' for re-integration into the proper location with the memory limitation that currently exists?

 

It would be cheating and not very TI...

I imagine some sort of TIPI/RPi integration with a TI program could accomplish the feat, however, for the record, I'm not even remotely suggesting this route.    

 

In the end...

It appears to me a whole new text file format would have to be developed, which is probably not going to happen due to how few would actually use such a program, even if it could be developed.  But from a strictly utilitarian purpose, it would be awesome to be able to write, import and maintain large documents on the TI without worrying about hitting a size limit.

 

A discussion on the feasibility and possible methods would be fascinating though...

I am looking at making an editor that reads the text file into a temp. virtual memory file.  The editor will operate on the buffers and the system will swap in chunks of the file and update the disk records as needed.  I am cheating because this functionality is part of Forth typically, but there is no law preventing it from being implemented in ASM or C.

 

This sounds like what was happening with your TRS-80 file system.

[atrax27407]

That sounds just like the 80-col editor in F'WEB with an AVPC or EVPC card.

[+InsaneMultitasker]

Paging and partial buffering sounds all well and good but the challenge is that as soon as you add or remove sufficient lines (or characters) to shift the file content by one or more sectors, you are for all intents and purposes changing the entire file from that point forward.  Eventually you will need to re-write the file from the point of change or in entirety. 

[Tursi]

True, that does become a challenge for an editor versus a simple reader. I guess my trick would be to treat the disk file as an actual swap file rather than trying to maintain it as a text file, so that you can separate the index to the lines from the line data itself. However that means to make the file actually usable as standard text you'd need an export function.... good call.