ivop Posted February 2, 2021 Share Posted February 2, 2021 10 hours ago, gnusto said: There's no way PAQ8 would run an Atari (1288MB of memory used for this lol) but that should be the lowest you can get this for comparison in a *general compressor* - PAQ is a consistent winner in compression efficiency among math fiends who are fascinated by this stuff and always testing. Yep, that's Prediction by Partial Matching (PPM) I mentioned earlier My xz example also uses a lot of memory. Around 900MB Quote Link to comment Share on other sites More sharing options...
Irgendwer Posted February 2, 2021 Share Posted February 2, 2021 On 2/1/2021 at 2:46 PM, xxl said: At this point, neither the compression speed nor the size of the buffer or library matters. There is a theoretical view on this matter too: You could create a library with every A8 production, images, sounds, code etc. created now or in the future. This way, you could easily address the result with an e.g. 8 byte key. You "just" have a huge dictionary for the "compression"... Found this perspective also interesting during studies. Quote Link to comment Share on other sites More sharing options...
Mark2008 Posted February 2, 2021 Share Posted February 2, 2021 Fun topic, I'm sure this is just senility, but I will share, that I decided to google on compression methods. And I was reading about the usual collection of algorithms, example lzma (7z), and then the article went into discussing image compression, including newer neural network based compression. One of which was Generative Adversarial Network based compression. OK, here is the funny thing, my tired eyes picked up on a sentence fragment "GAN-based compression can produce higher quality images." So...here is what I was thinking, you have lossy compression right, and lossless compression. And now "additive" compression. That can now produce higher quality images (than the original)....haha...which of course is not what they meant at all. Damn, my science fiction future was looking sweet. 1 Quote Link to comment Share on other sites More sharing options...
xxl Posted February 2, 2021 Author Share Posted February 2, 2021 18 minutes ago, Irgendwer said: You could create a library with every A8 production hahah, shortcut graphics could also be converted to fonts and compressed separately the image map and fonts something that G2F does ... but here you need to compress various data. 8 minutes ago, Mark2008 said: So...here is what I was thinking, you have lossy compression right, and lossless compression. And now "additive" compression. watch out ... someday there will be additive CODE algorithms... that will make the game Commando out of the Who Dare Wins II game code 1 Quote Link to comment Share on other sites More sharing options...
+gnusto Posted February 2, 2021 Share Posted February 2, 2021 10 hours ago, xxl said: that much memory is needed to decompress this file? : D Actually yes! This build of paq is symmetric in that way, but probably horribly inefficient. It's not really production code. Quote Link to comment Share on other sites More sharing options...
Sheddy Posted February 2, 2021 Share Posted February 2, 2021 slightly better deflate result: zopfli --i4000 --deflate conan.gfx 1,598 conan.gfx.deflate Quote Link to comment Share on other sites More sharing options...
xxl Posted February 2, 2021 Author Share Posted February 2, 2021 if anyone is interested in the unShrinkler decompression procedure for Atari 6502, I have shared it here: https://xxl.atari.pl/shrinkler-decompressor/ 6 3 Quote Link to comment Share on other sites More sharing options...
ivop Posted February 3, 2021 Share Posted February 3, 2021 48 minutes ago, xxl said: if anyone is interested in the unShrinkler decompression procedure for Atari 6502, I have shared it here: https://xxl.atari.pl/shrinkler-decompressor/ Nice code. Fun to see how you keep Y at 0 whenever it needs to be 0, and it's 0 again after the mulitply. And there's no ldy #0 anywhere in the code Not a geat fan of the @ usage for branches (@- and @-1) and rol @ (and I don't like it that I can't type a single @ on atariage without triggering the username lookup ) But as I said, nice code. Looking forward to where this goes in terms of usage in the future. 2 Quote Link to comment Share on other sites More sharing options...
xxl Posted February 6, 2021 Author Share Posted February 6, 2021 Fox joined in optimizing the code which resulted in another speedup. https://xxl.atari.pl/shrinkler-decompressor/ 7 Quote Link to comment Share on other sites More sharing options...
matosimi Posted February 11, 2021 Share Posted February 11, 2021 On 2/6/2021 at 4:10 PM, xxl said: Fox joined in optimizing the code which resulted in another speedup. https://xxl.atari.pl/shrinkler-decompressor/ 20 bytes in zero page and no buffer? do the 16bytes (except _src,_dst) have to be initialized to zeros on run? Quote Link to comment Share on other sites More sharing options...
xxl Posted February 11, 2021 Author Share Posted February 11, 2021 no, you can easily see that most of these ZP can be moved into code or behind decompressor code. there should only be 4 bytes left on page zero - dst and tabs no. the procedure itself initializes what is needed. buffers yes - in this version 6 pages (in fact 3 but the lower byte tabs is treated as a counter) - the buffer initializes itself. Quote Link to comment Share on other sites More sharing options...
Sheddy Posted February 13, 2021 Share Posted February 13, 2021 (edited) Just having a look at the source. "Jan”, "you", "constant". Macros? What do they do please? Edited February 13, 2021 by Sheddy Quote Link to comment Share on other sites More sharing options...
xxl Posted February 13, 2021 Author Share Posted February 13, 2021 7 minutes ago, Sheddy said: "Jan”, "you", "constant" ??? there are no such words. maybe turn off the automatic www translator maybe it breaks something. Quote Link to comment Share on other sites More sharing options...
Sheddy Posted February 13, 2021 Share Posted February 13, 2021 Ah yes. Thanks! I hadn't noticed it had translated. That 6502 is much more readable in the original Polish now. Quote Link to comment Share on other sites More sharing options...
Irgendwer Posted March 8, 2021 Share Posted March 8, 2021 On 2/6/2021 at 4:10 PM, xxl said: Fox joined in optimizing the code which resulted in another speedup. To lazy to try it out myself, so how much faster is the optimized version in the "Conan-case"? (over 500 frames vs. ?) Quote Link to comment Share on other sites More sharing options...
xxl Posted March 8, 2021 Author Share Posted March 8, 2021 the first version that was practically launched to see if it works took $22C frames for this picture. My later optmalizations gave $1E3, while Fox optimizations increased the performance to $1DE and have shortened the code (we are talking about the same picture all the time) I updated the page because I noticed that 4 labels were missing: https://xxl.atari.pl/shrinkler-decompressor/ 2 1 Quote Link to comment Share on other sites More sharing options...
xxl Posted March 24, 2021 Author Share Posted March 24, 2021 (edited) another big upgrade, more mature code, this time speeding up by two frames on the test program (image decompression) - bravo Fox. (code size $14E bytes) Edited March 24, 2021 by xxl 4 Quote Link to comment Share on other sites More sharing options...
elmer Posted March 26, 2021 Share Posted March 26, 2021 While this is all interesting in that there are definitely some cases where decompression time doesn't matter, I believe that those circumstances are rare, and that users generally don't like to be kept waiting (too much). From what I can see, nobody has yet mentioned the new-kid-on-the-block ... ZX0. Current testing show this to be as-fast or faster to decompress than APLIB, but with better compression ... so 2x or 3x as fast as Exomizer. lzsa: 1811 (lzsa.exe -f2 -r) apl: 1655 ZX0: 1625 EXO: 1537 (exomizer raw -E) 3 Quote Link to comment Share on other sites More sharing options...
xxl Posted March 26, 2021 Author Share Posted March 26, 2021 Great! https://github.com/einar-saukas/ZX0/tree/main/z80 I must check it out, it looks like a fairly simple algorithm. I write the result in the table in the first post. Quote Link to comment Share on other sites More sharing options...
xxl Posted March 26, 2021 Author Share Posted March 26, 2021 (edited) this ZX0 is pretty cool simple code conversion (if I have some time, I will optimize it) it can be significantly speeded up by replacing the copying procedure, I didn't touch the X register, it will optimize well. dzx0_standard lda #$ff sta offset sta offset+1 ldy #$00 sty len sty len+1 lda #$80 ; Literal (copy next N bytes from compressed file) ; 0 Elias(length) byte[1] byte[2] ... byte[N] dzx0s_literals jsr dzx0s_elias pha cop0 lda (compressed_data),y sta (decompressing),y inw decompressing inw compressed_data dew len lda len ora len+1 bne cop0 pla asl @ bcs dzx0s_new_offset ; Copy from last offset (repeat N bytes from last offset) ; 0 Elias(length) jsr dzx0s_elias dzx0s_copy pha lda decompressing clc adc offset sta copysrc lda decompressing+1 adc offset+1 sta copysrc+1 cop1 lda (copysrc),y sta (decompressing),y inw decompressing inw copysrc dew len lda len ora len+1 bne cop1 pla asl @ bcc dzx0s_literals ; Copy from new offset (repeat N bytes from new offset) ; 1 Elias(MSB(offset)) LSB(offset) Elias(length-1) dzx0s_new_offset jsr dzx0s_elias pha php lda #$00 sec sbc len sta offset+1 bne @+ plp pla rts ; koniec @ plp lda (compressed_data),y sta offset inw compressed_data ror offset+1 ror offset sty len+1 lda #1 sta len pla bcs @+ jsr dzx0s_elias_backtrack @ inw len jmp dzx0s_copy dzx0s_elias inc len ; inw ? dzx0s_elias_loop asl @ bne dzx0s_elias_skip lda (compressed_data),y inw compressed_data rol @ dzx0s_elias_skip bcc dzx0s_elias_backtrack rts dzx0s_elias_backtrack asl @ rol len rol len+1 jmp dzx0s_elias_loop zx0-gfx.obx Edited March 26, 2021 by xxl 2 Quote Link to comment Share on other sites More sharing options...
tane Posted March 26, 2021 Share Posted March 26, 2021 Can this be added to Super Packer ?? Quote Link to comment Share on other sites More sharing options...
xxl Posted March 26, 2021 Author Share Posted March 26, 2021 a bit of optimization, and it's almost as fast as the LZ4 https://xxl.atari.pl/zx0-decompressor/ 3 Quote Link to comment Share on other sites More sharing options...
xxl Posted March 26, 2021 Author Share Posted March 26, 2021 (edited) and a bootloader that loads ZX0 packed binaries I sent a request to Tebe, SuperPacker will receive this functionality ... maybe bootloader-zx0.atr Edited March 26, 2021 by xxl 1 Quote Link to comment Share on other sites More sharing options...
tane Posted March 27, 2021 Share Posted March 27, 2021 (edited) If I had something to say about to improve functionality of Super Packer, it would be to split the depacker and the compressed data in 2 different segments. So that, the depacker could be used several times as a function. It would save space. I think the only parameters required would be the memory addresses of the compressed data and the target to be decompressed. Such 4 bytes should be inside of the segment of the depacker, being possible to be modified with the program as required. At this moment, for every compressed segment the depacker is duplicated. Edited March 27, 2021 by tane Quote Link to comment Share on other sites More sharing options...
xxl Posted March 27, 2021 Author Share Posted March 27, 2021 10 minutes ago, tane said: At this moment, for every compressed segment the depacker is duplicated. no if you are using the "SE" version Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.