13 replies to this topic

[Bryan]

I was just thinking about all the different game menus and file compressors out there and wondered how well the two could be combined. What if a short loader-DOS with sector decompression were written so that the games could be written to the disk in a compressed format that the included mini-DOS could expand on the fly, giving increased capacity without the need to include a de-packer (and its delays) with each file. A utility could be used (either on the Atari, or PC) to add files to and extract files from the disk image. The only tricky part I can think of is making room for the buffers needed without breaking too many games.

[Rybags]

Packers don't really make the executable much bigger.  Probably under 300 bytes.

The problem like you say is that there's no "one fits all" solution - the depacker has to live somewhere and won't necessarily work with every file.

On top of that there's the loader itself so all up we're probably talking near 1/2 K that needs to not get overwritten.

With the menu I did years back, I put part of it in the IOCB area and might have even used the printer buffer.  I guess if you were to use part of the bottom half of the stack, printer buffer and IOCB area you'd fit about 300 bytes which should take care of the depacker at least.

I've not taken the cassette buffer into consideration - you need a sector buffer and most execuatables won't touch it during a load so it's ideal.

Of course additional to all these problems is larger sector sizes which bloats out the requirements more.


Finally, I don't worry much about file size any more.  I prefer a quick load to smaller size.  Given the large storage we have available with IDE devices and SIO2xx devices it's preferable e.g. to have a 40K file that loads/runs in 15 seconds to a 20K file that loads/depacks/runs in 20 seconds.

[Bryan]

Yeah, it's more of intriguing exercize than anything. I still like the idea of making floppy-sized game disks but I'd like to get the most games possible on a disk. I suppose the utility could let you specify the areas used by the compression routine for problem files, and it could examine what areas are required by the file headers. It could even use the area under the OS if it was XL/XE specific.

The other fun part would be making a really cool menu. Just something for the back burner.

[Rybags]

I guess potentially the thing could be developed to be relocatable.  A well written program as such might only have a dozen or so operands that need adjustment.

Another downside though is that the files are then totally dependant on the loader and you'd not be able to use them under Dos or a different loader.

I also question the savings.  On my old 130K game floppies I probably averaged 9 games per side.
Compressing all the files with self-contained depack would probably increase that to about 15.  The saving of having a unified depack for all would save 14 occurrences which might only add up to a bit over 4K.

[Bryan]

Well, the unified depacker is mainly to try and write one that works on-the-fly and makes sectors available at a rate comparable to regular loading. It's to get rid of the depacking phase.

[Rybags]

Problem with depack on the fly is that real floppy media will grind to a crawl.  You only really get optimal load speed when doing "bust I/O", ie the absolute minimum of processing between sectors.

Packers also tend to work much better on larger blocks of data.  If you're only dealing with a few hundred bytes then there's less chance of repeated strings, so less efficient compression.

It's probably worth a look at SuperPacker from Tebe, just to see how much overhead is involved with the depack routines.

[xxl]

loader with decompressor is not good idea because games may write hiscores or other data and such loader/IOlibrary should by extremly short, do not use zero page nor OS, interrupt...

so... here you go: http://www.atari.org...=141318#p141318

for ataridos, dd, standard speed, sio drive

[rdemming]

Since game loader disks are only meant for loading executables and not for access by a regular DOS, maybe you could consider a new file system that supports long file names, sub directories and increases the 64 files per directory limit. A PC program could generate an ATR for such format and  an utility for the Atari could also be made to transfer files from a regular DOS disk to the game loader disk.

The file limit for regular DOSes is annoying when you use 16MB hard disk partitions and an additional text file for long file names such as MyPicoDos uses is not very user-friendly.

Robert

[Bryan]

I was thinking some more about this...

1. From what little I've read about compression algorithms, It's possible to write a routine that takes far more time to compress than de-compress. I don't care how long they take to compress, as long decompressing them is fairly efficient. If the compression tool works with the disk as an image, it could actually compress repetition across all files rather than just the current one. Sounds more like a job for a PC program, or an A8 with a big RAMdisk.

2. Isn't the CPU sitting idle much of the time during a 19.2K load?

3. I don't really care if write access is denied, or if a small area of the disk can be designated as free for uncompressed file writes for high scores. IMO, the loader doesn't have to handle every possibility.

Anyway, it's not something I have time for right now. I just wanted some opinions on it.

[Rybags]

You can do plenty during idle time with normal SIO speeds.

The problem is, a standard speed SIO loader becomes a huge step backwards.  Even with the best possible mix of compression and depack speed, you'd probably only match 2x SIO loading speeds.

I guess you could do a turbo speed loader.  The higher speeds use polling rather than IRQs anyway.  But then the problem becomes that you have little CPU cycles left over for depack.

I suppose the ideal medium would be something that can depack on the fly and maintain a "medium fast" SIO speed.  But having all this would cost a fair bit in memory so then you run into the problem that many files would corrupt the loader.

[phaeron]

I think the main problem you'd have with decompression is memory for the required block sizes, not CPU time.

There are two main forms of compression that can be used or combined. The first type works on probability of symbols appearing, particularly Huffman tree coding. Encoding is not much slower than decoding, but you need memory for the tree and a big enough block size to amortize the tree encoding. There is also a tradeoff between the unpacked tree size and decoding speed as more memory allows use of tables to decode multiple bits at a time. Alternatively, a fairly compact form is to store a list of counts by length and a sorted symbol table, at the cost of more work to decode per symbol. The second type of compression is sliding window compression, which encodes repeats of previously seen text. Run-length encoding can be seen as a degenerate form of this where the distance is always 0. This type of compression is highly asymmetric, with encoding being slow and decoding being much faster, but the main downside is that you either need to be decompressing into a contiguous block of memory or need enough memory on the side to hold the sliding window for copying references.

I used a form of LZ compression in my Acid5200 test to squeeze everything onto a 32K cart. It's pretty fast, but there's a visible pause when it decompresses and it needs 4K for the history buffer. The ratios are all over the place, with some tests compressing 3:1 and the CPU test not really compressing at all. You'd really need a block size bigger than one sector to make this work in practice, or else the ratios just won't be high enough to make this worthwhile.

[Thorsten Günther]

rdemming, on Mon Feb 6, 2012 3:10 AM, said:

Since game loader disks are only meant for loading executables and not for access by a regular DOS, maybe you could consider a new file system that supports long file names, sub directories and increases the 64 files per directory limit. A PC program could generate an ATR for such format and  an utility for the Atari could also be made to transfer files from a regular DOS disk to the game loader disk.

The file limit for regular DOSes is annoying when you use 16MB hard disk partitions and an additional text file for long file names such as MyPicoDos uses is not very user-friendly.

There already is such a tool - as a shell command for Windows: <http://www.horus.com...i/#tools-win32>

A proper command to generate a game ATR would be "dir2atr -P -b MyPicoDos405 [Filename].atr [Directory]"

There's even some switches to choose the MyPicoDos settings (e.g. high speed disabled or enabled after booting), just read the manual.
IMHO, this is much more comfortable than editing the PICONAME.TXT file manually.

BTW: the file packer displaying a countdown while depacking seems to be the most efficient one, but I don't know which one this is. I suppose someone here is able to enlighten me!?

Thorsten

Edited by Thorsten Günther, Mon Feb 20, 2012 2:48 PM.

[Fox-1 / mnx]

Thorsten Günther, on Mon Feb 20, 2012 2:46 PM, said:

There already is such a tool - as a shell command for Windows: <http://www.horus.com...i/#tools-win32>

A proper command to generate a game ATR would be "dir2atr -P -b MyPicoDos405 [Filename].atr [Directory]"

This won't do for HDD's on real XL/XE.

[CharlieChaplin]

Thorsten Günther, on Mon Feb 20, 2012 2:46 PM, said:

BTW: the file packer displaying a countdown while depacking seems to be the most efficient one, but I don't know which one this is. I suppose someone here is able to enlighten me!?

Thorsten

Well,

I guess you mean the Code3-Cruncher with countdown (there are also versions with snow-like screens while depacking). Afaik there are some more packers which use a countdown when depacking...

Besides, Code3-Cruncher has a good packing efficiency, but not the best one on the A8, DJ-Packer is a little better, Deflate/Inflate routine by FoX is even better and errrm, Superpacker (Exomizer) by TeBe beats them all. But this Superpacker is a PC program (easily confused with Superpacker on the A8 by Bewesoft)...

-Andreas Koch.

P.S.: The newest package of Superpacker by TeBe (Version 3.3) also comes with a deflat7z.exe but it seems not to be implemented right now (unlike the deflat.exe which is already implemented into Superpacker)...

Edited by CharlieChaplin, Mon Feb 20, 2012 5:00 PM.