Loader for 'hybrid' binary files

[xxl]

I have added a hybrid binary files decompressor to the bootloader. Such files can be created from any binary file by compressing its selected blocks, e.g. with Super Packer ( http://madteam.atari8.info/uzytki/sp.7z Of course, the bootloader can also load standard binary files. 

You can implement such a loader on your SIO2xx device - the bootloader code in the attachment. The size of the loader is a standard bootsector - 384 bytes

Below is an example:

281 977 bytes - an uncompressed BombJack game

121 814 bytes - BombJack a 'standard' self-extracting binary file

98 463 bytes - BombJack with compressed LZ4 blocks to run from the loader

 

example of operation in a .ATR file

xBoot-lz4.obx bj.atr BJ_v1_5.xex Bomb_Jack_15.xex BJ.xex

Edited November 26, 2019 by xxl

[rensoup]

It's nice to have decompression on the fly but to be honest, I'm already compressing pretty much everything in Pop with inflate.

 

What would be cool is a speed fix for real 1050 drives... but I guess not too many people are interested in that.

[rensoup]

Just curious, would it be possible to remove the disk writing code to save some bytes ?

[baktra]

I would have a question on how to prepare the binary load files with Super Packer for the hybrid loader.

Is it as follows?

 

1. Take non-compressed binary load file

2. Open the file in Super Packer

3. Select all segments and compress with LZ4

4. Remove segment with LZ4 depacker

5. Save to a new binary load file

 

I am also surprised by the contents of the BJ.XEX file when I open it in Super Packer and some segment's last addresses are lower then first addresses. Why is that?

 

Thank you.

 

 

 

[xxl]

no....

first of all, download the latest version of SuperPacker - only this one works correctly for such files.

http://madteam.atari8.info/index.php?prod=uzytki#sp

configure: use smallz4 (so far probably the most efficient of all lz4)

press LZ4 SE

1. Take non-compressed binary file

2. Open the file in Super Packer

3. Select ONE segment and compress with pack sgement

4. repeat step 3 for all segments you want to pack

5. Save to a new binary file

you don't have to delete anything because no additional segments will be added

 

now:

save the file to ATR and initialize with this program:

http://xxl.atari.pl/download/xBOOT_Initializer_LZ4.xex

 

first post programs are out of date

 

files created in this way can also be loaded with LiteDOS 3 (it loads binary files with compressed blocks)

 

[baktra]

28 minutes ago, xxl said:

no....

first of all, download the latest version of SuperPacker - only this one works correctly for such files.

http://madteam.atari8.info/index.php?prod=uzytki#sp

configure: use smallz4 (so far probably the most efficient of all lz4)

press LZ4 SE

1. Take non-compressed binary file

2. Open the file in Super Packer

3. Select ONE segment and compress with pack sgement

4. repeat step 3 for all segments you want to pack

5. Save to a new binary file

you don't have to delete anything because no additional segments will be added

 

now:

save the file to ATR and initialize with this program:

http://xxl.atari.pl/download/xBOOT_Initializer_LZ4.xex

 

first post programs are out of date

 

files created in this way can also be loaded with LiteDOS 3 (it loads binary files with compressed blocks)

 

Thank you, now it is crystal clear.

[mr-atari]

LiteDOS 3 will be available next year.

 

Still coding on certain routines.....

Exporting LiteDUP to a separate file.

 

If you want to try/test, a work-in-progress is available on request if you pm or e mail me.  

[baktra]

I tried to compress few binary load files with LZ4 SE in Super Packer.

 

The LZ4 SE compressed segments have their last address changed to 0. I take it that the last address of zero indicates the segment is compressed and its last address is given by the length of the decompressed data.

 

Is that correct that after that zero last address, the LZ4 block format follows, or is the format somehow different?

[xxl]

end data address = 0 identifies the compressed block, the next byte identifies the type of compressor and further the compressed data

[mr-atari]

 

Clowns and Balloons can be compressed nicely, loading time goes from 18 to 14 seconds.

?

[Poison]

It looks good, but it should run separately in normal DOS. When it shrinks from 40kb to 20kb it's cool, but if I can't run it, it's useless. I don't make sense for unrunable short files. But for true all-disk games, yes. I see no reason why one 20kb game should occupy the entire floppy disk.

[xxl]

files can be run from DOS, which supports new functionality. maybe the dos you use is unusable.

 

LiteDOS and xB support compressed executable files.

[Poison]

Yes, but I have DOS MDDADOK in cart. I do not need to use another one. Iam using it for 30 years

[xxl]

maybe it's time to update. instead of complaining

[Jacques]

3 hours ago, Poison said:

It looks good, but it should run separately in normal DOS. When it shrinks from 40kb to 20kb it's cool, but if I can't run it, it's useless. I don't make sense for unrunable short files.

I couldn't agree more, smaller compressed files? Why not, as long as they're runnable without need to use some Frankensteins and run from typical DOSes.

 

Quote

files can be run from DOS, which supports new functionality. maybe the dos you use is unusable.

 

LiteDOS and xB support compressed executable files.

 

Yeah, let's forget running file games/programs from AtariDOS, MyDOS, SpartaDOS X what we were doing since 80's, let's switch to marginal LiteDOS and xB, because we can save few kilos when it doesn't really matter in XXI century. Not to mention decompression is slower when loading binaries from parallel devices instead of SIO and with Ultraspeed 3x SIO, no one would really give a sh** about saving frictions of a second, too.

Edited December 28, 2019 by Jacques
typo

[xxl]

you don't want to upgrade the software but you have no problem updating the hardware :-)?
you prefer to solve problems on the hardware way - if something works slowly you buy a modern fast device - take into account that there are people who prefer to solve problems (even size or speed) by software way - without the need to invest in equipment ?

Edited December 28, 2019 by xxl

[Jacques]

I don't perceive xB or LiteDOS an upgrade to standards of AtariDOS/MyDOS/DOSII+/SpartaDOS X.

They may be highly specialized tools aimed at leaving free as much RAM as possible, but not as versatile and something I'd choose for typical use.

 

 

 

Edited December 28, 2019 by Jacques

[xxl]

standard hypocrisy ? if you have to load the relocated (Sparta) file, do you protest that you can't load it with DOS 2.5 bla bla? No, you politely load it with DOS that supports it ?

 

Similarly here - this is a function that other DOS do not have.

Do you want programs not developed for many years to have new functionality?
Do you want NO new functionality to appear because you don't want to use more modern software?

 

? do not answer. I already know. ?

[Jacques]

SDX is modern, constantly developed, have great documentation and provides gazillion of possibilities and ways of configuration.

Fine by me, xB nor LiteDOS is no choice just because of the advantage they could depack Frankenstein-compressed binaries and are short in RAM-usage.

 

But you can still advertise and promote like the best thing since sliced bread

Edited December 28, 2019 by Jacques

[xxl]

7 minutes ago, Jacques said:

SDX is modern, constantly developed, have great documentation and provides gazillion of possibilities and ways of configuration.

except the functionality discussed here ? probably this is your "resistance".

I hope that in some way (constantly develope) Sparta will have support for binary files with packed blocks and you won't have to complain.

 

?

[Faicuai]

10 hours ago, xxl said:

except the functionality discussed here ? probably this is your "resistance".

I hope that in some way (constantly develope) Sparta will have support for binary files with packed blocks and you won't have to complain.

 

?

Well, in all fairness, today's world of Atari users is essentially split in two:

 

1. Host computers with access to large / mass storage devices (e.g. "Hard Drives" or SD-driven Cartridges with high-speed loaders)

2. Host computers that, at best, have access to std. or accelerated SIO-based storage (eg. SIO-to-USB, SDrive, Floppy Drives, etc.)

 

One thing that I am sure about is that, as title-files and productions get larger and larger, #2 above eventually becomes burdensome and disruptive.... to the point that it is much, much better for everyone (users and developers) to simply upgrade HW or just purchase dedicated HD storage device (SIDE II., Ultimate or AVG cart, etc.)

 

Under SDX, I load PANG (Final version, 190K file) directly from command prompt in 6.19 secs. From BIOS loader (Ultimate or Incognito), same file in 5.19 secs. 

 

That is a HUGE difference in speed, in relative terms... Sufficient enough to forget dumb-ass SIO-driven storage, for once and all, with these particular type of (large) projects.

 

 

[pps]

xBIOS is a great tool and it still grows up!

[+gnusto]

22 hours ago, xxl said:

files can be run from DOS, which supports new functionality. maybe the dos you use is unusable

 

First, thanks for continuing to breath life into the 8bit. We all love these machines, and we all support anyone developing on them.

 

A suggestion for the above, you will REALLY help your adoption if you stick a very small header on your compressed file, that prints out "Please load using compressed block support" or similar, when it is loaded from a standard DOS. You could start using different extensions, but those have a way of being changed without your control, and if you think of a future where people freely exchange files of both types, there will be those who get compressed ones and are using for instance Atari DOS (which is never going away) and will just be confused.

[xxl]

An interesting idea, but in my opinion, it is not necessary to increase the file size. Does a similar message appear to DOS2.5 users when they try to load a file in which one of the blocks at the end has a header supported by Sparta?
But it would be good to mark such files with the extension? Compresed Executable? *.cxe?

[miker]

Really too hard to add something that exports packed program with depacker?