zagon Posted November 3, 2005 Share Posted November 3, 2005 The next version of Exomizer (2.0) will support the 8-bit Ataris as a stand alone decruncher target. This support is now implemented in a beta version and I am looking for Atari beta testers. This tool has previously only supported the Commodore line of 8-bit machines so I am very interested to get some feedback from Atari people. The exomizer is a cross-cruncher that can crunch files for in memory decrunching, from file decrunching and stand alone decrunching of runnable files. There's a website for version 1.x that brieflyexplains what it's all about here http://web.comhem.se/~u13114991/exo/ and the beta version with Atari support is downloadable from here http://web.comhem.se/~u13114991/exo/exo20beta2.zip Example usage: $ exomizer sfx sys -t168 boinxx9.obx -o x.xex The download includes binaries for win32 and dos. It also includes the complete sourcecode which is compilable on Linux as well. Please note that it is a beta release so there's probably a few rough corners still present. Quote Link to comment Share on other sites More sharing options...
fox Posted November 4, 2005 Share Posted November 4, 2005 Could you shortly describe the compression algorithm used? Quote Link to comment Share on other sites More sharing options...
Heaven/TQA Posted November 4, 2005 Share Posted November 4, 2005 Fox, how does exo compares to our cruncher which i am using in Boinxx?? Quote Link to comment Share on other sites More sharing options...
fox Posted November 4, 2005 Share Posted November 4, 2005 I don't know know what kind of compression exo uses, nor what routine did you use in Boinxx. Quote Link to comment Share on other sites More sharing options...
Heaven/TQA Posted November 4, 2005 Share Posted November 4, 2005 i am using the compression routines we have used in numen. Quote Link to comment Share on other sites More sharing options...
zagon Posted November 4, 2005 Author Share Posted November 4, 2005 Exomizer uses a variant of LZ77 and tries to calculate an optimal encoding of the lengths and offsets. It also allows lengts that are longer than the offsets in order to compress repeating patterns and rle-sequences. Please check out the rawdecrs/exodecr.c decruncher sourcecode in the archive for more details. Quote Link to comment Share on other sites More sharing options...
fox Posted November 4, 2005 Share Posted November 4, 2005 So, the code that Heaven used is a standard zlib-compatible inflate routine: dynamic Huffman-coded LZ77 with 32KB window (same as in ZIP). As I understand, exo does some kind of Huffman only for offsets and length, not uncompressed bytes, right? My old FlashPack did only LZ77 2- and 3-byte sequences in 127-byte window + RLE and compressed marks of uncompressed bytes (a block of 8 uncompressed bytes is marked with a single bit). Quote Link to comment Share on other sites More sharing options...
zagon Posted November 4, 2005 Author Share Posted November 4, 2005 Unlike zlib inflate/deflate, exomizer doesn't do any encoding of uncompressed bytes but it does compress sequences down to the length of one byte to compensate. Another difference compared to zlib inflate/deflate is that the encoding is static and not dynamic. This makes the decruncher simpler and probably faster than zlib deflate. However the compression result is often better for typical 8-bit computer data. Quote Link to comment Share on other sites More sharing options...
Wrathchild Posted November 4, 2005 Share Posted November 4, 2005 Great stuff - I'd looked at Exomizer as while back (Apr'05) and liked it - http://www.atariage.com/forums/index.php?showtopic=68641 Quick test on a mulit-part file F:\FUN\Atari800Win>exomizer sfx sys -t168 elite.com -o elite.xex filename: "elite.com", loading from $0A00 to $BBDD Propagating xex runad 1F61. crunching from $0A00 to $BBDD Phase 1: Instrumenting file ----------------------------- Length of indata: 45533 bytes. [building.directed.acyclic.graph.building.directed.acyclic.graph.] Instrumenting file, done. Phase 2: Calculating encoding ----------------------------- pass 1: optimizing .. [finding.cheapest.path.finding.cheapest.path.finding.cheapest.pat] size 179310.0 bits ~22414 bytes pass 2: optimizing .. [finding.cheapest.path.finding.cheapest.path.finding.cheapest.pat] size 179045.0 bits ~22381 bytes pass 3: optimizing .. [finding.cheapest.path.finding.cheapest.path.finding.cheapest.pat] size 179045.0 bits ~22381 bytes Calculating encoding, done. Phase 3: Generating output file ------------------------------ Encoding: 1001123235578C0C,2122,2123345556677789,33334567789AABDE Length of crunched data: 22409 bytes. Target is self-decrunching Atari 400/800 XL/XE executable, jmp address $1F61. So original = 33,050 bytes, new xex = 22,707 bytes Compression ratio = 68.7%, ~two thirds - pretty good going. My plain boring huffman encoder only gave me 98% and that's without any decompressing code Makes a nice study for those interested... Regards, Mark Compress.zip Quote Link to comment Share on other sites More sharing options...
Heaven/TQA Posted November 5, 2005 Share Posted November 5, 2005 i am using this depack code: * this is the depacker written by Fox/Taquart ; Uncompress data stored in the DEFLATE format. ; ; DEFLATE is a popular compression format, used e.g. in ZIP and GZIP archives, ; and the ZLIB library. The original description of this format ; is available in RFC (Request for Comments) 1951 in the file ; ftp://www.ietf.org/rfc/rfc1951.txt. ; Both compressed and uncompressed data must completely fit in the memory. ; As the compressed data is read sequentially and only once, it is possible ; for the compressed and uncompressed data to overlap, i.e. the data being ; uncompressed can be stored in place of some compressed data which has been ; already read. There is no general rule, but practically both data may ; overlap almost entirely, with the uncompressed data ending just a few ; bytes before the end of the compressed data. ; If you want to modify this code to support non-continuous memory areas ; for the output, you should note that the uncompressed data is used as ; a look-behind buffer, up to 32K. ; As the code is not self-modifying, it can be put in ROM. ; This source code is written in the 6502 assembly language, ; using syntax of xasm (http://xasm.atari.org). ; The one-character comments I use are: ; '!' branch always ; '-' C flag is zero (useful for adc) ; '+' C flag is set (useful for sbc) ; At the end of this file you can find source code of a simple C program ; that performs DEFLATE compression using the popular ZLIB library. ; ; Permission is granted to anyone to use this code for any purpose, including ; commercial applications, and redistribute it freely in its original form. ; ; Author: Piotr Fusik <fox@scene.pl> ; Last modified: 11 Oct 2003 * Constants ; Whether to support the new DEFLATE64, introduced as the compression method 9 ; in PKZIP 4.0, released Nov 2000. This format is very uncommon (not even ; supported by ZLIB) and doesn't seem to be useful on 6502 systems, since ; the main change is the 64K look-behind buffer. It's here just for fun! :-) USE_DEFLATE64 equ 0 ; Maximum length of a Huffman code. MAX_BITS equ 15 ; All Huffman trees are stored in the bitsCount, bitsPointer_l ; and bitsPointer_h arrays. There may be two trees: the literal/length tree ; and the distance tree, or just one - the temporary tree. ; Index in the mentioned arrays for the beginning of the literal/length tree ; or the temporary tree. PRIMARY_TREE equ 0 ; Index in the mentioned arrays for the beginning of the distance tree. DISTANCE_TREE equ MAX_BITS ; Size of each array. TREES_SIZE equ 2*MAX_BITS * Page zero ; (public) Pointer to the compressed data. inputPointer equ zpage ; 2 bytes ; (public) Pointer to the uncompressed data. outputPointer equ zpage+2 ; 2 bytes ; Local variables. ; As far as there is no conflict, same memory locations are used ; for different variables. inflateDynamicBlock_cnt equ zpage+4 ; 1 byte inflateCodes_src equ zpage+4 ; 2 bytes buildHuffmanTree_src equ zpage+4 ; 2 bytes getNextLength_last equ zpage+4 ; 1 byte getNextLength_index equ zpage+5 ; 1 byte buildHuffmanTree_ptr equ zpage+6 ; 2 bytes fetchCode_ptr equ zpage+6 ; 2 bytes getBits_tmp equ zpage+6 ; 1 byte moveBlock_len equ zpage+8 ; 2 bytes inflateDynamicBlock_np equ zpage+8 ; 1 byte inflateDynamicBlock_nd equ zpage+9 ; 1 byte getBit_hold equ zpage+10; 1 byte * Code * (public) inflate ; Decompress the DEFLATE data starting from the address stored in inputPointer ; to the memory starting from the address stored in outputPointer. depacker mvy #1 getBit_hold bne inflate_2 ! inflate_1 jsr inflate_3 inflate_2 ; Get a bit of EOF and two bits of block type ldx #3 lda #0 jsr getBits lsr @ bcc inflate_1 inflate_3 lsr @ bne inflateDynamicBlock ; Note: inflateDynamicBlock may assume that A = 1 bcs inflateFixedBlock ; Note: inflateCopyBlock may assume that C = 0 * inflateCopyBlock ; Decompress a 'stored' data block. inflateCopyBlock ; Ignore bits until byte boundary mvy #1 getBit_hold ; Get 16-bit length ldx #inputPointer mva (0,x) moveBlock_len mva (inputPointer),y moveBlock_len+1 ; Skip the length and one's complement of it lda #4 adc:sta inputPointer - scc:inc inputPointer+1 * moveBlock ; Copy block of length moveBlock_len from (0,x) to the output. moveBlock ldy moveBlock_len beq moveBlock_1 ldy #0 inc moveBlock_len+1 moveBlock_1 mva (0,x) (outputPointer),y inw 0,x inw outputPointer dec moveBlock_len bne moveBlock_1 dec moveBlock_len+1 bne moveBlock_1 rts * inflateFixedBlock ; Decompress a Huffman-coded data block with default Huffman trees ; (defined by the DEFLATE format): ; literalCodeLength :144 dta 8 ; :112 dta 9 ; endCodeLength dta 7 ; lengthCodeLength :23 dta 7 ; :6 dta 8 ; distanceCodeLength :30+2*USE_DEFLATE64 dta 5+DISTANCE_TREE ; :2 dta 8 ; (two 8-bit codes from the primary tree are not used). inflateFixedBlock ldx #159+USE_DEFLATE64 stx distanceCodeLength+32+2*USE_DEFLATE64 lda #8 inflateFixedBlock_1 sta literalCodeLength-1,x sta literalCodeLength+159+USE_DEFLATE64-1,x- bne inflateFixedBlock_1 ldx #112 inc:rne literalCodeLength+144-1,x- ldx #24 dec:rne endCodeLength-1,x- ldx #30+2*USE_DEFLATE64 lda #5+DISTANCE_TREE sta:rne distanceCodeLength-1,x- beq inflateCodes ! * inflateDynamicBlock ; Decompress a Huffman-coded data block, reading Huffman trees first. inflateDynamicBlock ; numberOfPrimaryCodes = 257 + getBits(5) ldx #5 ; lda #1 jsr getBits sta inflateDynamicBlock_np ; numberOfDistanceCodes = 1 + getBits(5) ldx #5 lda #1+29+1 jsr getBits sta inflateDynamicBlock_nd ; numberOfTemporaryCodes = 4 + getBits(4) lda:tax #4 jsr getBits sta inflateDynamicBlock_cnt ; Get lengths of temporary codes in the order stored in tempCodeLengthOrder txa:tay #0 inflateDynamicBlock_1 ldx #3 ; A = 0 jsr getBits ; does not change Y inflateDynamicBlock_2 ldx tempCodeLengthOrder,y sta literalCodeLength,x lda #0 iny cpy inflateDynamicBlock_cnt bcc inflateDynamicBlock_1 cpy #19 bcc inflateDynamicBlock_2 ror literalCodeLength+19 + ; Build the tree for temporary codes jsr buildHuffmanTree ; Use temporary codes to get lengths of literal/length and distance codes ldx #0 ldy #1 stx getNextLength_last inflateDynamicBlock_3 jsr getNextLength sta literalCodeLength,x+ bne inflateDynamicBlock_3 inflateDynamicBlock_4 jsr getNextLength inflateDynamicBlock_5 sta endCodeLength,x+ cpx inflateDynamicBlock_np bcc inflateDynamicBlock_4 lda #0 cpx #1+29 bcc inflateDynamicBlock_5 inflateDynamicBlock_6 jsr getNextLength cmp #0 seq:adc #DISTANCE_TREE-1 + sta endCodeLength,x+ cpx inflateDynamicBlock_nd bcc inflateDynamicBlock_6 ror endCodeLength,x + * inflateCodes ; Decompress a data block using given Huffman trees. inflateCodes jsr buildHuffmanTree inflateCodes_1 jsr fetchPrimaryCode bcs inflateCodes_2 ; Literal code sta (outputPointer),0 inc outputPointer bne inflateCodes_1 inc outputPointer+1 bcc inflateCodes_1 ! ; End of block inflateCodes_ret rts inflateCodes_2 beq inflateCodes_ret ; Restore a block from the look-behind buffer jsr getValue sta moveBlock_len tya jsr getBits sta moveBlock_len+1 ldx #DISTANCE_TREE jsr fetchCode jsr getValue sec eor #$ff adc outputPointer sta inflateCodes_src php tya jsr getBits plp eor #$ff adc outputPointer+1 sta inflateCodes_src+1 ldx #inflateCodes_src jsr moveBlock beq inflateCodes_1 ! * buildHuffmanTree ; Build Huffman trees basing on code lengths (in bits) ; stored in the *CodeLength arrays. ; A byte with its highest bit set marks the end. buildHuffmanTree mwa #literalCodeLength buildHuffmanTree_src ; Clear bitsCount and bitsPointer_l ldy #2*TREES_SIZE+1 lda #0 sta:rne bitsCount-1,y- beq buildHuffmanTree_3 ! ; Count number of codes of each length buildHuffmanTree_2 tax inc bitsPointer_l,x iny bne buildHuffmanTree_3 inc buildHuffmanTree_src+1 buildHuffmanTree_3 lda (buildHuffmanTree_src),y bpl buildHuffmanTree_2 ; Calculate a pointer for each length ldx #0 lda #<sortedCodes ldy #>sortedCodes clc buildHuffmanTree_4 sta bitsPointer_l,x tya:sta bitsPointer_h,x lda bitsPointer_l+1,x adc bitsPointer_l,x - scc:iny inx cpx #TREES_SIZE bcc buildHuffmanTree_4 mva #>literalCodeLength buildHuffmanTree_src+1 ldy #0 bcs buildHuffmanTree_9 ! ; Put codes into their place in the sorted array buildHuffmanTree_6 beq buildHuffmanTree_7 tax mva bitsPointer_l-1,x buildHuffmanTree_ptr mva bitsPointer_h-1,x buildHuffmanTree_ptr+1 tya ldy:inc bitsCount-1,x sta (buildHuffmanTree_ptr),y tay buildHuffmanTree_7 iny bne buildHuffmanTree_9 inc buildHuffmanTree_src+1 ldx #MAX_BITS-1 mva:rpl bitsCount,x literalCount,x- buildHuffmanTree_9 lda (buildHuffmanTree_src),y bpl buildHuffmanTree_6 rts * getNextLength ; Decode next code length using temporary codes. getNextLength stx getNextLength_index dey bne getNextLength_1 ; Fetch a temporary code jsr fetchPrimaryCode ; Temporary code 0..15: put this length ldy #1 cmp #16 bcc getNextLength_2 ; Temporary code 16: repeat last length 3 + getBits(2) times ; Temporary code 17: put zero length 3 + getBits(3) times ; Temporary code 18: put zero length 11 + getBits(7) times tay ldx tempExtraBits-16,y lda tempBaseValue-16,y jsr getBits cpy #17 tay txa #0 bcs getNextLength_2 getNextLength_1 lda getNextLength_last getNextLength_2 sta getNextLength_last ldx getNextLength_index rts * fetchPrimaryCode ; Read a code basing on the primary tree. fetchPrimaryCode ldx #PRIMARY_TREE * fetchCode ; Read a code from input basing on the tree specified in X. ; Return low byte of this code in A. ; For the literal/length tree, the C flag is set if the code is non-literal. fetchCode lda #0 fetchCode_1 jsr getBit rol @ inx sub bitsCount-1,x bcs fetchCode_1 adc bitsCount-1,x - cmp literalCount-1,x sta fetchCode_ptr ldy bitsPointer_l-1,x mva bitsPointer_h-1,x fetchCode_ptr+1 lda (fetchCode_ptr),y rts * getValue ; Decode low byte of a value (length or distance), basing on the code in A. ; The result is the base value for this code plus some bits read from input. getValue tay ldx lengthExtraBits-1,y lda:pha lengthBaseValue_l-1,y lda:tay lengthBaseValue_h-1,y pla * getBits ; Read X-bit number from the input and add it to A. ; Increment Y if overflow. ; If X > 8, read only 8 bits. ; On return X holds number of unread bits: X = (X > 8 ? X - 8 : 0); getBits cpx #0 beq getBits_ret pha mva #-1 getBits_tmp pla getBits_1 jsr getBit bcc getBits_2 sbc getBits_tmp + scc:iny getBits_2 dex beq getBits_ret asl getBits_tmp bmi getBits_1 getBits_ret rts * getBit ; Read a single bit from input, return it in the C flag. getBit lsr getBit_hold bne getBit_ret pha sty getBit_hold lda (inputPointer),0 ldy getBit_hold inw inputPointer ror @ + sta getBit_hold pla getBit_ret rts * Arrays for the temporary codes. ; Order, in which lengths of the temporary codes are stored. tempCodeLengthOrder dta b(16,17,18,0,8,7,9,6,10,5,11,4,12,3,13,2,14,1,15) ; Base values. tempBaseValue dta b(3,3,11) ; Number of extra bits to read. tempExtraBits dta b(2,3,7) * Arrays for the length and distance codes. ; Base values. lengthBaseValue_l dta l(3,4,5,6,7,8,9,10,11,13,15,17,19,23,27,31,35,43) dta l(51,59,67,83,99,115,131,163,195,227) :!USE_DEFLATE64 dta l(258) :USE_DEFLATE64 dta l(3) distanceBaseValue_l dta l(1,2,3,4,5,7,9,13,17,25,33,49,65,97,129,193,257,385) dta l(513,769,1025,1537,2049,3073,4097,6145,8193,12289,16385,24577) :USE_DEFLATE64 dta l(32769,49153) lengthBaseValue_h dta h(3,4,5,6,7,8,9,10,11,13,15,17,19,23,27,31,35,43) dta h(51,59,67,83,99,115,131,163,195,227) :!USE_DEFLATE64 dta h(258) :USE_DEFLATE64 dta h(3) distanceBaseValue_h dta h(1,2,3,4,5,7,9,13,17,25,33,49,65,97,129,193,257,385) dta h(513,769,1025,1537,2049,3073,4097,6145,8193,12289,16385,24577) :USE_DEFLATE64 dta h(32769,49153) ; Number of extra bits to read. lengthExtraBits dta b(0,0,0,0,0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3,4,4) dta b(4,4,5,5,5,5) :!USE_DEFLATE64 dta b(0) :USE_DEFLATE64 dta b(16) distanceExtraBits dta b(0,0,0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,8,8,9,9,10,10) dta b(11,11,12,12,13,13) :USE_DEFLATE64 dta b(14,14) ; Number of literal codes of each length in the primary tree ; (MAX_BITS bytes, overlap with literalCodeLength). literalCount * Data for building the primary tree. ; Lengths of literal codes. literalCodeLength org *+256 ; Length of the end code. endCodeLength org *+1 ; Lengths of length codes. lengthCodeLength org *+29 * Data for building the distance tree. ; Lengths of distance codes. distanceCodeLength org *+30+2*USE_DEFLATE64 ; For two unused codes in the fixed trees and an 'end' mark org *+3 * The Huffman trees. ; Number of codes of each length. bitsCount org *+TREES_SIZE ; Pointers to sorted codes of each length. bitsPointer_l org *+TREES_SIZE+1 bitsPointer_h org *+TREES_SIZE ; Sorted codes. sortedCodes org *+256+1+29+30+2*USE_DEFLATE64+2 and attached the packer deflater.zip Quote Link to comment Share on other sites More sharing options...
Heaven/TQA Posted November 5, 2005 Share Posted November 5, 2005 zagon, how can i use exomizer to crunch level data? it seems just working with files holding run adresses? Quote Link to comment Share on other sites More sharing options...
zagon Posted November 5, 2005 Author Share Posted November 5, 2005 You use the 'mem' sub-command instead of 'sfx'. This command produces a .prg file so if you need to convert it to a plain file you'll have to remove the two first bytes of it (the load address). There's some documentation about this sub-command in the exo20info.txt file and a (c64 targeted) example of this in the exodecrs folder. (the one using the exodecrunch.s decruncher source file.) The examples in this folder are using gnu-make and the cc65 tool-chain. (The makefile has a hardcoded path to the exomizer binary so you'll probably have to change it to where it is located on your system.) Quote Link to comment Share on other sites More sharing options...
fox Posted November 5, 2005 Share Posted November 5, 2005 Length of indata: 45533 bytes. So original = 33,050 bytes, new xex = 22,707 bytes Was the original 33050 or 45533 bytes long? Can you "gzip -9" the original file and add 800 bytes for the depack routine, for a quick comparison? Or even better, create GZip archive ("Ultra" compression level) using 7-Zip on Windows? Quote Link to comment Share on other sites More sharing options...
fox Posted November 5, 2005 Share Posted November 5, 2005 Unlike zlib inflate/deflate, exomizer doesn't do any encoding of uncompressed bytes but it does compress sequences down to the length of one byte to compensate. Do you mean you support one-byte LZ77 sequences? Another difference compared to zlib inflate/deflate is that the encoding is static and not dynamic. This makes the decruncher simpler and probably faster than zlib deflate. My zlib decompress routine isn't actually that slow, thanks to clever lookup tables. Actually it's several times faster than most of the other decompressors on Atari. I've used it in Numen for background decompression while the effects are running. Quote Link to comment Share on other sites More sharing options...
Heaven/TQA Posted November 5, 2005 Share Posted November 5, 2005 it's interesting that Boinxx9 got packed again with such a high rate... all data files (3x music, 4 level data) were already packed with Fox' packer... Quote Link to comment Share on other sites More sharing options...
Heaven/TQA Posted November 5, 2005 Share Posted November 5, 2005 it works and the exomizer depacker is quite fast as well... compared to superpacker... boinxx9.zip x.zip Quote Link to comment Share on other sites More sharing options...
zagon Posted November 5, 2005 Author Share Posted November 5, 2005 Was the original 33050 or 45533 bytes long? Both are right, the original xex-file is probably 33050 and the total length of data when all xex-segments are loaded into memory from the lowest to the highest used address is 45533 bytes. Do you mean you support one-byte LZ77 sequences? yes. Another difference from zlib is how the offsets work. The offsets of Exomizer points to the 'wrong' end of the sequences, the end furthest away. But this has the advantage of supporting sequences that can encode repeating patterns of one or more bytes like this: (offset 1, length 2345) == rle sequence. (offset 4, length 16) == four byte pattern repeated 4 times. Your zlib decompress routine sounds interesting, I'll have to take a look at it. Quote Link to comment Share on other sites More sharing options...
Wrathchild Posted November 5, 2005 Share Posted November 5, 2005 Was the original 33050 or 45533 bytes long? Both are right, the original xex-file is probably 33050 and the total length of data when all xex-segments are loaded into memory from the lowest to the highest used address is 45533 bytes. Yep, that's what I'd suspected, fortunately I don't think there were too many 'blank' areas between the segments. One question I had though, relating to cart development. Currently I have some routines that RLE a file and additionally gives you an offset table for every 256 bytes (1 page), though that is configurable - e.g. you could do 128 bytes instead. So, compression can't span a block and also compressed data shouldn't span a ROM bank boundry, typically 8K. See the code in the Gauntlet rom thread for an example: http://www.atariage.com/forums/index.php?showtopic=67762 So I'm interested in knowing if Exomiser compression could be used to in some way - e.g. maybe using a wrapper (e.g. script) to split an input file into parts and then compress them individually and then construct the header and compressed datafile from. Regards, Mark Quote Link to comment Share on other sites More sharing options...
zagon Posted November 6, 2005 Author Share Posted November 6, 2005 So I'm interested in knowing if Exomiser compression could beused to in some way - e.g. maybe using a wrapper (e.g. script) to split an input file into parts and then compress them individually and then construct the header and compressed datafile from. It should be doable, but it will be a bit tricky. However, I would suggest crunching bigger chunks of data than 128 or 256 bytes in order to improve crunch result. I would also suggest using the streaming decruncher, the one with the cyclic buffer, to decrunch the chunk just far enough untill the wanted byte range is reached. However, the cyclic buffer uses some memory and the chunk size will be a tradeoff between crunch result and the time it will take to access a single byte range. It also means that there has to be a two level index structure, one to tell in which cruched chunk the wanted byte range is in, and another to tell the offset of the byte range in the decrunched chunk data. Quote Link to comment Share on other sites More sharing options...
Wrathchild Posted November 6, 2005 Share Posted November 6, 2005 The overall aim would to be to use the decoder as a replacement 'get sector' routine. So rather than analysis being done on a per sector basis, it could be done against the whole file, but then the compression of each sector would be done using those rules - not as effecient I guess, but good enough. Quote Link to comment Share on other sites More sharing options...
zagon Posted November 6, 2005 Author Share Posted November 6, 2005 Those sectors, are they always read in some kind of order? If so, you could store the byte ranges in the chunks in the order they are read so that multiple sector reads could be served by just one decrunch. In other words: Is there a file-layer abstraction hiding behind those sector reads? If so, it could would be possible to take advantage of that to further improve compression result and performance. By the way, In Exomizer I now use address $2c00 for loading the sfx crunched file. Is this address safe for all kinds of dos versions? Is there some other address that is generally agreed to be safe that would be better to use? Quote Link to comment Share on other sites More sharing options...
fox Posted November 7, 2005 Share Posted November 7, 2005 Another difference from zlib is how the offsets work.The offsets of Exomizer points to the 'wrong' end of the sequences, the end furthest away. But this has the advantage of supporting sequences that can encode repeating patterns of one or more bytes like this: (offset 1, length 2345) == rle sequence. (offset 4, length 16) == four byte pattern repeated 4 times. It's the same in zlib. I repeat, can someone do a comparison of compression results: exo vs zlib? Maybe I can find the uncompressed "Drunk Chessboard" somewhere. It was 24KB, and 16 KB when compressed with FlashPack. The depack routine was something like 80 bytes long and supported LZ77+RLE+changing destination memory addresses. I think you can't beat its speed. * Depack routine for FlashPack 2.1 * By Fox of Taquart, 5th May 1997 * Average speed: 30 kB/sec * 4 bytes on page 0 ff equ $fc bt equ $fd ad equ $fe * Do not start from here!!! dep1 tax beq ret lda #$7f dep2 bcc *+3 inx inx sta ad dep3 lda (ad),y put sta $8080,y iny bne dep4 inc ad+1 inc put+2 dep4 dex bne dep3 asl bt bne dep7 asl ff bne dep5 * Routine starts here! start equ * sec jsr get rol @ sta ff dep5 lda #1 bcc dep6 jsr get rol @ dep6 sta bt dep7 jsr get ldx #1 bcc put lsr @ bne dep2 jsr get bcs dep1 tay jsr get sta ad+1 sta put+2 bcc dep7 ! * Set address of packed data here! * (or make your own "get" routine) get lda $ffff inc get+1 bne ret inc get+2 ret rts end of code Quote Link to comment Share on other sites More sharing options...
zagon Posted November 7, 2005 Author Share Posted November 7, 2005 It's the same in zlib. Hm, it seems I have to do my homework better It was 24KB, and 16 KB when compressed with FlashPack. The depack routine was something like 80 bytes long and supported LZ77+RLE+changing destination memory addresses. I think you can't beat its speed. That's impressive, I probably can't Quote Link to comment Share on other sites More sharing options...
tebe Posted June 11, 2018 Share Posted June 11, 2018 (edited) Exomizer 3.0 https://bitbucket.org/magli143/exomizer/wiki/Home better / faster / shorter use switch -P0 for compatybility with exomizer 2.0 p.s. Super Packer work only with exomizer 2.0 Edited June 11, 2018 by tebe 7 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.