Jump to content
Sign in to follow this  
zagon

Exomizer beta

Recommended Posts

The next version of Exomizer (2.0) will support the 8-bit Ataris as a stand alone decruncher target. This support is now implemented in a beta version and I am looking for Atari beta testers. This tool has previously only supported the Commodore line of 8-bit machines so I am very interested to get some feedback from Atari people.

 

The exomizer is a cross-cruncher that can crunch files for in memory decrunching, from file decrunching and stand alone decrunching of runnable files.

 

There's a website for version 1.x that brieflyexplains what it's all about here http://web.comhem.se/~u13114991/exo/ and the beta version with Atari support is downloadable from here http://web.comhem.se/~u13114991/exo/exo20beta2.zip

 

Example usage:

$ exomizer sfx sys -t168 boinxx9.obx -o x.xex

 

The download includes binaries for win32 and dos. It also includes the complete sourcecode which is compilable on Linux as well.

 

Please note that it is a beta release so there's probably a few rough corners still present.

Share this post


Link to post
Share on other sites

I don't know know what kind of compression exo uses, nor what routine did you use in Boinxx.

Share this post


Link to post
Share on other sites

Exomizer uses a variant of LZ77 and tries to calculate an optimal encoding of

the lengths and offsets.

 

It also allows lengts that are longer than the offsets in order to compress repeating patterns and rle-sequences.

 

Please check out the rawdecrs/exodecr.c decruncher sourcecode in the archive for more details.

Share this post


Link to post
Share on other sites

So, the code that Heaven used is a standard zlib-compatible inflate routine: dynamic Huffman-coded LZ77 with 32KB window (same as in ZIP). As I understand, exo does some kind of Huffman only for offsets and length, not uncompressed bytes, right?

 

My old FlashPack did only LZ77 2- and 3-byte sequences in 127-byte window

+ RLE and compressed marks of uncompressed bytes (a block of 8 uncompressed bytes is marked with a single bit).

Share this post


Link to post
Share on other sites

Unlike zlib inflate/deflate, exomizer doesn't do any encoding of uncompressed bytes but it does compress sequences down to the length of one byte to compensate.

Another difference compared to zlib inflate/deflate is that the encoding is static and not dynamic. This makes the decruncher simpler and probably faster than zlib deflate.

However the compression result is often better for typical 8-bit computer data.

Share this post


Link to post
Share on other sites

Great stuff - I'd looked at Exomizer as while back (Apr'05) and liked it -

http://www.atariage.com/forums/index.php?showtopic=68641

 

Quick test on a mulit-part file

 

F:\FUN\Atari800Win>exomizer sfx sys -t168 elite.com -o elite.xex
filename: "elite.com", loading from $0A00 to $BBDD
Propagating xex runad 1F61.
crunching from $0A00 to $BBDD
Phase 1: Instrumenting file
-----------------------------
Length of indata: 45533 bytes.
[building.directed.acyclic.graph.building.directed.acyclic.graph.]
Instrumenting file, done.

Phase 2: Calculating encoding
-----------------------------
pass 1: optimizing ..
[finding.cheapest.path.finding.cheapest.path.finding.cheapest.pat]
 size 179310.0 bits ~22414 bytes
pass 2: optimizing ..
[finding.cheapest.path.finding.cheapest.path.finding.cheapest.pat]
 size 179045.0 bits ~22381 bytes
pass 3: optimizing ..
[finding.cheapest.path.finding.cheapest.path.finding.cheapest.pat]
 size 179045.0 bits ~22381 bytes
Calculating encoding, done.

Phase 3: Generating output file
------------------------------
Encoding: 1001123235578C0C,2122,2123345556677789,33334567789AABDE
Length of crunched data: 22409 bytes.
Target is self-decrunching Atari 400/800 XL/XE executable,
jmp address $1F61.

 

So original = 33,050 bytes, new xex = 22,707 bytes

 

Compression ratio = 68.7%, ~two thirds - pretty good going.

 

My plain boring huffman encoder only gave me 98% and that's

without any decompressing code :D

 

Makes a nice study for those interested...

 

Regards,

Mark

Compress.zip

Share this post


Link to post
Share on other sites

i am using this depack code:

* this is the depacker written by Fox/Taquart

; Uncompress data stored in the DEFLATE format.
;
; DEFLATE is a popular compression format, used e.g. in ZIP and GZIP archives,
; and the ZLIB library.  The original description of this format
; is available in RFC (Request for Comments) 1951 in the file
; ftp://www.ietf.org/rfc/rfc1951.txt.
; Both compressed and uncompressed data must completely fit in the memory.
; As the compressed data is read sequentially and only once, it is possible
; for the compressed and uncompressed data to overlap, i.e. the data being
; uncompressed can be stored in place of some compressed data which has been
; already read.  There is no general rule, but practically both data may
; overlap almost entirely, with the uncompressed data ending just a few
; bytes before the end of the compressed data.
; If you want to modify this code to support non-continuous memory areas
; for the output, you should note that the uncompressed data is used as
; a look-behind buffer, up to 32K.
; As the code is not self-modifying, it can be put in ROM.
; This source code is written in the 6502 assembly language,
; using syntax of xasm (http://xasm.atari.org).
; The one-character comments I use are:
; '!' branch always
; '-' C flag is zero (useful for adc)
; '+' C flag is set (useful for sbc)
; At the end of this file you can find source code of a simple C program
; that performs DEFLATE compression using the popular ZLIB library.
;
; Permission is granted to anyone to use this code for any purpose, including
; commercial applications, and redistribute it freely in its original form.
;
; Author: Piotr Fusik <[email protected]>
; Last modified: 11 Oct 2003


* Constants

; Whether to support the new DEFLATE64, introduced as the compression method 9
; in PKZIP 4.0, released Nov 2000.  This format is very uncommon (not even
; supported by ZLIB) and doesn't seem to be useful on 6502 systems, since
; the main change is the 64K look-behind buffer.  It's here just for fun! :-)
USE_DEFLATE64           equ	0

; Maximum length of a Huffman code.
MAX_BITS                equ	15

; All Huffman trees are stored in the bitsCount, bitsPointer_l
; and bitsPointer_h arrays.  There may be two trees: the literal/length tree
; and the distance tree, or just one - the temporary tree.

; Index in the mentioned arrays for the beginning of the literal/length tree
; or the temporary tree.
PRIMARY_TREE            equ	0

; Index in the mentioned arrays for the beginning of the distance tree.
DISTANCE_TREE           equ	MAX_BITS

; Size of each array.
TREES_SIZE              equ	2*MAX_BITS


* Page zero

; (public) Pointer to the compressed data.
inputPointer            equ	zpage   ; 2 bytes

; (public) Pointer to the uncompressed data.
outputPointer           equ	zpage+2 ; 2 bytes

; Local variables.
; As far as there is no conflict, same memory locations are used
; for different variables.

inflateDynamicBlock_cnt equ	zpage+4 ; 1 byte
inflateCodes_src        equ	zpage+4 ; 2 bytes
buildHuffmanTree_src    equ	zpage+4 ; 2 bytes
getNextLength_last      equ	zpage+4 ; 1 byte
getNextLength_index     equ	zpage+5 ; 1 byte

buildHuffmanTree_ptr    equ	zpage+6 ; 2 bytes
fetchCode_ptr           equ	zpage+6 ; 2 bytes
getBits_tmp             equ	zpage+6 ; 1 byte

moveBlock_len           equ	zpage+8 ; 2 bytes
inflateDynamicBlock_np  equ	zpage+8 ; 1 byte
inflateDynamicBlock_nd  equ	zpage+9 ; 1 byte

getBit_hold             equ	zpage+10; 1 byte


* Code

* (public) inflate
; Decompress the DEFLATE data starting from the address stored in inputPointer
; to the memory starting from the address stored in outputPointer.
depacker mvy	#1	getBit_hold
bne	inflate_2	!
inflate_1
jsr	inflate_3
inflate_2
; Get a bit of EOF and two bits of block type
ldx	#3
lda	#0
jsr	getBits
lsr	@
bcc	inflate_1
inflate_3
lsr	@
bne	inflateDynamicBlock
; Note: inflateDynamicBlock may assume that A = 1
bcs	inflateFixedBlock
; Note: inflateCopyBlock may assume that C = 0

* inflateCopyBlock
; Decompress a 'stored' data block.
inflateCopyBlock
; Ignore bits until byte boundary
mvy	#1	getBit_hold
; Get 16-bit length
ldx	#inputPointer
mva	(0,x)	moveBlock_len
mva	(inputPointer),y	moveBlock_len+1
; Skip the length and one's complement of it
lda	#4
adc:sta	inputPointer	-
scc:inc	inputPointer+1

* moveBlock
; Copy block of length moveBlock_len from (0,x) to the output.
moveBlock
ldy	moveBlock_len
beq	moveBlock_1
ldy	#0
inc	moveBlock_len+1
moveBlock_1
mva	(0,x)	(outputPointer),y
inw	0,x
inw	outputPointer
dec	moveBlock_len
bne	moveBlock_1
dec	moveBlock_len+1
bne	moveBlock_1
rts

* inflateFixedBlock
; Decompress a Huffman-coded data block with default Huffman trees
; (defined by the DEFLATE format):
; literalCodeLength  :144 dta 8
;                    :112 dta 9
; endCodeLength           dta 7
; lengthCodeLength   :23  dta 7
;                    :6   dta 8
; distanceCodeLength :30+2*USE_DEFLATE64  dta 5+DISTANCE_TREE
;                    :2   dta 8
; (two 8-bit codes from the primary tree are not used).
inflateFixedBlock
ldx	#159+USE_DEFLATE64
stx	distanceCodeLength+32+2*USE_DEFLATE64
lda	#8
inflateFixedBlock_1
sta	literalCodeLength-1,x
sta	literalCodeLength+159+USE_DEFLATE64-1,x-
bne	inflateFixedBlock_1
ldx	#112
inc:rne	literalCodeLength+144-1,x-
ldx	#24
dec:rne	endCodeLength-1,x-
ldx	#30+2*USE_DEFLATE64
lda	#5+DISTANCE_TREE
sta:rne	distanceCodeLength-1,x-
beq	inflateCodes	!

* inflateDynamicBlock
; Decompress a Huffman-coded data block, reading Huffman trees first.
inflateDynamicBlock
; numberOfPrimaryCodes = 257 + getBits(5)
ldx	#5
;	lda	#1
jsr	getBits
sta	inflateDynamicBlock_np
; numberOfDistanceCodes = 1 + getBits(5)
ldx	#5
lda	#1+29+1
jsr	getBits
sta	inflateDynamicBlock_nd
; numberOfTemporaryCodes = 4 + getBits(4)
lda:tax	#4
jsr	getBits
sta	inflateDynamicBlock_cnt
; Get lengths of temporary codes in the order stored in tempCodeLengthOrder
txa:tay	#0
inflateDynamicBlock_1
ldx	#3      ; A = 0
jsr	getBits ; does not change Y
inflateDynamicBlock_2
ldx	tempCodeLengthOrder,y
sta	literalCodeLength,x
lda	#0
iny
cpy	inflateDynamicBlock_cnt
bcc	inflateDynamicBlock_1
cpy	#19
bcc	inflateDynamicBlock_2
ror	literalCodeLength+19	+
; Build the tree for temporary codes
jsr	buildHuffmanTree

; Use temporary codes to get lengths of literal/length and distance codes
ldx	#0
ldy	#1
stx	getNextLength_last
inflateDynamicBlock_3
jsr	getNextLength
sta	literalCodeLength,x+
bne	inflateDynamicBlock_3
inflateDynamicBlock_4
jsr	getNextLength
inflateDynamicBlock_5
sta	endCodeLength,x+
cpx	inflateDynamicBlock_np
bcc	inflateDynamicBlock_4
lda	#0
cpx	#1+29
bcc	inflateDynamicBlock_5
inflateDynamicBlock_6
jsr	getNextLength
cmp	#0
seq:adc	#DISTANCE_TREE-1	+
sta	endCodeLength,x+
cpx	inflateDynamicBlock_nd
bcc	inflateDynamicBlock_6
ror	endCodeLength,x	+

* inflateCodes
; Decompress a data block using given Huffman trees.
inflateCodes
jsr	buildHuffmanTree
inflateCodes_1
jsr	fetchPrimaryCode
bcs	inflateCodes_2
; Literal code
sta	(outputPointer),0
inc	outputPointer
bne	inflateCodes_1
inc	outputPointer+1
bcc	inflateCodes_1	!
; End of block
inflateCodes_ret
rts
inflateCodes_2
beq	inflateCodes_ret
; Restore a block from the look-behind buffer
jsr	getValue
sta	moveBlock_len
tya
jsr	getBits
sta	moveBlock_len+1
ldx	#DISTANCE_TREE
jsr	fetchCode
jsr	getValue
sec
eor	#$ff
adc	outputPointer
sta	inflateCodes_src
php
tya
jsr	getBits
plp
eor	#$ff
adc	outputPointer+1
sta	inflateCodes_src+1
ldx	#inflateCodes_src
jsr	moveBlock
beq	inflateCodes_1	!

* buildHuffmanTree
; Build Huffman trees basing on code lengths (in bits)
; stored in the *CodeLength arrays.
; A byte with its highest bit set marks the end.
buildHuffmanTree
mwa	#literalCodeLength	buildHuffmanTree_src
; Clear bitsCount and bitsPointer_l
ldy	#2*TREES_SIZE+1
lda	#0
sta:rne	bitsCount-1,y-
beq	buildHuffmanTree_3	!
; Count number of codes of each length
buildHuffmanTree_2
tax
inc	bitsPointer_l,x
iny
bne	buildHuffmanTree_3
inc	buildHuffmanTree_src+1
buildHuffmanTree_3
lda	(buildHuffmanTree_src),y
bpl	buildHuffmanTree_2
; Calculate a pointer for each length
ldx	#0
lda	#<sortedCodes
ldy	#>sortedCodes
clc
buildHuffmanTree_4
sta	bitsPointer_l,x
tya:sta	bitsPointer_h,x
lda	bitsPointer_l+1,x
adc	bitsPointer_l,x	-
scc:iny
inx
cpx	#TREES_SIZE
bcc	buildHuffmanTree_4
mva	#>literalCodeLength	buildHuffmanTree_src+1
ldy	#0
bcs	buildHuffmanTree_9	!
; Put codes into their place in the sorted array
buildHuffmanTree_6
beq	buildHuffmanTree_7
tax
mva	bitsPointer_l-1,x	buildHuffmanTree_ptr
mva	bitsPointer_h-1,x	buildHuffmanTree_ptr+1
tya
ldy:inc	bitsCount-1,x
sta	(buildHuffmanTree_ptr),y
tay
buildHuffmanTree_7
iny
bne	buildHuffmanTree_9
inc	buildHuffmanTree_src+1
ldx	#MAX_BITS-1
mva:rpl	bitsCount,x	literalCount,x-
buildHuffmanTree_9
lda	(buildHuffmanTree_src),y
bpl	buildHuffmanTree_6
rts

* getNextLength
; Decode next code length using temporary codes.
getNextLength
stx	getNextLength_index
dey
bne	getNextLength_1
; Fetch a temporary code
jsr	fetchPrimaryCode
; Temporary code 0..15: put this length
ldy	#1
cmp	#16
bcc	getNextLength_2
; Temporary code 16: repeat last length 3 + getBits(2) times
; Temporary code 17: put zero length 3 + getBits(3) times
; Temporary code 18: put zero length 11 + getBits(7) times
tay
ldx	tempExtraBits-16,y
lda	tempBaseValue-16,y
jsr	getBits
cpy	#17
tay
txa	#0
bcs	getNextLength_2
getNextLength_1
lda	getNextLength_last
getNextLength_2
sta	getNextLength_last
ldx	getNextLength_index
rts

* fetchPrimaryCode
; Read a code basing on the primary tree.
fetchPrimaryCode
ldx	#PRIMARY_TREE

* fetchCode
; Read a code from input basing on the tree specified in X.
; Return low byte of this code in A.
; For the literal/length tree, the C flag is set if the code is non-literal.
fetchCode
lda	#0
fetchCode_1
jsr	getBit
rol	@
inx
sub	bitsCount-1,x
bcs	fetchCode_1
adc	bitsCount-1,x	-
cmp	literalCount-1,x
sta	fetchCode_ptr
ldy	bitsPointer_l-1,x
mva	bitsPointer_h-1,x	fetchCode_ptr+1
lda	(fetchCode_ptr),y
rts

* getValue
; Decode low byte of a value (length or distance), basing on the code in A.
; The result is the base value for this code plus some bits read from input.
getValue
tay
ldx	lengthExtraBits-1,y
lda:pha	lengthBaseValue_l-1,y
lda:tay	lengthBaseValue_h-1,y
pla

* getBits
; Read X-bit number from the input and add it to A.
; Increment Y if overflow.
; If X > 8, read only 8 bits.
; On return X holds number of unread bits: X = (X > 8 ? X - 8 : 0);
getBits
cpx	#0
beq	getBits_ret
pha
mva	#-1	getBits_tmp
pla
getBits_1
jsr	getBit
bcc	getBits_2
sbc	getBits_tmp	+
scc:iny
getBits_2
dex
beq	getBits_ret
asl	getBits_tmp
bmi	getBits_1
getBits_ret
rts

* getBit
; Read a single bit from input, return it in the C flag.
getBit
lsr	getBit_hold
bne	getBit_ret
pha
sty	getBit_hold
lda	(inputPointer),0
ldy	getBit_hold
inw	inputPointer
ror	@	+
sta	getBit_hold
pla
getBit_ret
rts


* Arrays for the temporary codes.

; Order, in which lengths of the temporary codes are stored.
tempCodeLengthOrder
dta	b(16,17,18,0,8,7,9,6,10,5,11,4,12,3,13,2,14,1,15)

; Base values.
tempBaseValue
dta	b(3,3,11)

; Number of extra bits to read.
tempExtraBits
dta	b(2,3,7)


* Arrays for the length and distance codes.

; Base values.
lengthBaseValue_l
dta	l(3,4,5,6,7,8,9,10,11,13,15,17,19,23,27,31,35,43)
dta	l(51,59,67,83,99,115,131,163,195,227)
:!USE_DEFLATE64	dta	l(258)
:USE_DEFLATE64	dta	l(3)
distanceBaseValue_l
dta	l(1,2,3,4,5,7,9,13,17,25,33,49,65,97,129,193,257,385)
dta	l(513,769,1025,1537,2049,3073,4097,6145,8193,12289,16385,24577)
:USE_DEFLATE64	dta	l(32769,49153)
lengthBaseValue_h
dta	h(3,4,5,6,7,8,9,10,11,13,15,17,19,23,27,31,35,43)
dta	h(51,59,67,83,99,115,131,163,195,227)
:!USE_DEFLATE64	dta	h(258)
:USE_DEFLATE64	dta	h(3)
distanceBaseValue_h
dta	h(1,2,3,4,5,7,9,13,17,25,33,49,65,97,129,193,257,385)
dta	h(513,769,1025,1537,2049,3073,4097,6145,8193,12289,16385,24577)
:USE_DEFLATE64	dta	h(32769,49153)

; Number of extra bits to read.
lengthExtraBits
dta	b(0,0,0,0,0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3,4,4)
dta	b(4,4,5,5,5,5)
:!USE_DEFLATE64	dta	b(0)
:USE_DEFLATE64	dta	b(16)
distanceExtraBits
dta	b(0,0,0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,8,8,9,9,10,10)
dta	b(11,11,12,12,13,13)
:USE_DEFLATE64	dta	b(14,14)

; Number of literal codes of each length in the primary tree
; (MAX_BITS bytes, overlap with literalCodeLength).
literalCount


* Data for building the primary tree.

; Lengths of literal codes.
literalCodeLength
org	*+256
; Length of the end code.
endCodeLength
org	*+1
; Lengths of length codes.
lengthCodeLength
org	*+29


* Data for building the distance tree.

; Lengths of distance codes.
distanceCodeLength
org	*+30+2*USE_DEFLATE64
; For two unused codes in the fixed trees and an 'end' mark
org	*+3


* The Huffman trees.

; Number of codes of each length.
bitsCount
org	*+TREES_SIZE
; Pointers to sorted codes of each length.
bitsPointer_l
org	*+TREES_SIZE+1
bitsPointer_h
org	*+TREES_SIZE

; Sorted codes.
sortedCodes
org	*+256+1+29+30+2*USE_DEFLATE64+2


 

and attached the packer

deflater.zip

Share this post


Link to post
Share on other sites

You use the 'mem' sub-command instead of 'sfx'. This command produces a .prg file so if you need to convert it to a plain file you'll have to remove the two first bytes of it (the load address).

 

There's some documentation about this sub-command in the exo20info.txt file and a (c64 targeted) example of this in the exodecrs folder. (the one using the exodecrunch.s decruncher source file.) The examples in this folder are using gnu-make and the cc65 tool-chain. (The makefile has a hardcoded path to the exomizer binary so you'll probably have to change it to where it is located on your system.)

Share this post


Link to post
Share on other sites
 Length of indata: 45533 bytes.

So original = 33,050 bytes, new xex = 22,707 bytes

 

Was the original 33050 or 45533 bytes long?

Can you "gzip -9" the original file and add 800 bytes for the depack routine,

for a quick comparison?

Or even better, create GZip archive ("Ultra" compression level) using 7-Zip on Windows?

Share this post


Link to post
Share on other sites
Unlike zlib inflate/deflate, exomizer doesn't do any encoding of uncompressed bytes but it does compress sequences down to the length of one byte to compensate.

Do you mean you support one-byte LZ77 sequences? :o

Another difference compared to zlib inflate/deflate is that the encoding is static and not dynamic. This makes the decruncher simpler and probably faster than zlib deflate.

My zlib decompress routine isn't actually that slow, thanks to clever lookup tables.

Actually it's several times faster than most of the other decompressors on Atari.

I've used it in Numen for background decompression while the effects are running.

Share this post


Link to post
Share on other sites

it's interesting that Boinxx9 got packed again with such a high rate... all data files (3x music, 4 level data) were already packed with Fox' packer...

Share this post


Link to post
Share on other sites
Was the original 33050 or 45533 bytes long?

Both are right, the original xex-file is probably 33050 and the total length

of data when all xex-segments are loaded into memory from the lowest to the highest used address is 45533 bytes.

 

Do you mean you support one-byte LZ77 sequences? :-o

yes.

 

Another difference from zlib is how the offsets work.

The offsets of Exomizer points to the 'wrong' end of the sequences, the end furthest away. But this has the advantage of supporting sequences that can encode repeating patterns of one or more bytes like this:

(offset 1, length 2345) == rle sequence.

(offset 4, length 16) == four byte pattern repeated 4 times.

 

Your zlib decompress routine sounds interesting, I'll have to take a look at it. :)

Share this post


Link to post
Share on other sites
Was the original 33050 or 45533 bytes long?

Both are right, the original xex-file is probably 33050 and the total length

of data when all xex-segments are loaded into memory from the lowest to the highest used address is 45533 bytes.

Yep, that's what I'd suspected, fortunately I don't think there were too many 'blank' areas between the segments.

 

One question I had though, relating to cart development.

Currently I have some routines that RLE a file and additionally

gives you an offset table for every 256 bytes (1 page), though

that is configurable - e.g. you could do 128 bytes instead.

So, compression can't span a block and also compressed

data shouldn't span a ROM bank boundry, typically 8K.

See the code in the Gauntlet rom thread for an example:

http://www.atariage.com/forums/index.php?showtopic=67762

 

So I'm interested in knowing if Exomiser compression could be

used to in some way - e.g. maybe using a wrapper (e.g. script)

to split an input file into parts and then compress them individually

and then construct the header and compressed datafile from.

 

Regards,

Mark

Share this post


Link to post
Share on other sites
So I'm interested in knowing if Exomiser compression could be

used to in some way - e.g. maybe using a wrapper (e.g. script)

to split an input file into parts and then compress them individually

and then construct the header and compressed datafile from.

 

It should be doable, but it will be a bit tricky. However, I would suggest crunching bigger chunks of data than 128 or 256 bytes in order to improve crunch result. I would also suggest using the streaming decruncher, the one with the cyclic buffer, to decrunch the chunk just far enough untill the wanted byte range is reached.

 

However, the cyclic buffer uses some memory and the chunk size will be a tradeoff between crunch result and the time it will take to access a single byte range. It also means that there has to be a two level index structure, one to tell in which cruched chunk the wanted byte range is in, and another to tell the offset of the byte range in the decrunched chunk data.

Share this post


Link to post
Share on other sites

The overall aim would to be to use the decoder as a replacement 'get sector' routine. So rather than analysis being done on a per sector basis, it could be done against the whole file, but then the compression of each sector would be done using those rules - not as effecient I guess, but good enough.

Share this post


Link to post
Share on other sites

Those sectors, are they always read in some kind of order? If so, you could store the byte ranges in the chunks in the order they are read so that multiple sector reads could be served by just one decrunch.

 

In other words: Is there a file-layer abstraction hiding behind those sector reads?

If so, it could would be possible to take advantage of that to further improve compression result and performance.

 

By the way, In Exomizer I now use address $2c00 for loading the sfx crunched file.

Is this address safe for all kinds of dos versions? Is there some other address that is generally agreed to be safe that would be better to use?

Share this post


Link to post
Share on other sites
Another difference from zlib is how the offsets work.

The offsets of Exomizer points to the 'wrong' end of the sequences, the end furthest away. But this has the advantage of supporting sequences that can encode repeating patterns of one or more bytes like this:

(offset 1, length 2345) == rle sequence.

(offset 4, length 16) == four byte pattern repeated 4 times.

It's the same in zlib.

 

I repeat, can someone do a comparison of compression results: exo vs zlib?

 

Maybe I can find the uncompressed "Drunk Chessboard" somewhere.

It was 24KB, and 16 KB when compressed with FlashPack. The depack routine was something like 80 bytes long and supported LZ77+RLE+changing destination memory addresses. I think you can't beat its speed.

 

* Depack routine for FlashPack 2.1
* By Fox of Taquart, 5th May 1997
* Average speed: 30 kB/sec

* 4 bytes on page 0
ff equ $fc
bt equ $fd
ad equ $fe

* Do not start from here!!!
dep1 tax
beq ret
lda #$7f
dep2 bcc *+3
inx
inx
sta ad
dep3 lda (ad),y
put  sta $8080,y
iny
bne dep4
inc ad+1
inc put+2
dep4 dex
bne dep3
asl bt
bne dep7
asl ff
bne dep5
* Routine starts here!
start equ *
sec
jsr get
rol @
sta ff
dep5 lda #1
bcc dep6
jsr get
rol @
dep6 sta bt
dep7 jsr get
ldx #1
bcc put
lsr @
bne dep2
jsr get
bcs dep1
tay
jsr get
sta ad+1
sta put+2
bcc dep7 !
* Set address of packed data here!
* (or make your own "get" routine)
get lda $ffff
inc get+1
bne ret
inc get+2
ret rts

end of code

Share this post


Link to post
Share on other sites
It's the same in zlib.

 

Hm, it seems I have to do my homework better :ponder:

 

It was 24KB, and 16 KB when compressed with FlashPack. The depack routine was something like 80 bytes long and supported LZ77+RLE+changing destination memory addresses. I think you can't beat its speed.

 

That's impressive, I probably can't :)

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...