Jump to content
xxl

Most efficient compression for Atari

Recommended Posts

10 hours ago, gnusto said:

There's no way PAQ8 would run an Atari (1288MB of memory used for this lol) but that should be the lowest you can get this for comparison in a *general compressor* - PAQ is a consistent winner in compression efficiency among math fiends who are fascinated by this stuff and always testing.

Yep, that's Prediction by Partial Matching (PPM) I mentioned earlier :)

 

My xz example also uses a lot of memory. Around 900MB ;)

 

Share this post


Link to post
Share on other sites
On 2/1/2021 at 2:46 PM, xxl said:

At this point, neither the compression speed nor the size of the buffer or library matters.

There is a theoretical view on this matter too:

You could create a library with every A8 production, images, sounds, code etc. created now or in the future.

This way, you could easily address the result with an e.g. 8 byte key. You "just" have a huge dictionary for the "compression"...

Found this perspective also interesting during studies.

Share this post


Link to post
Share on other sites

Fun topic, I'm sure this is just senility, but I will share, that I decided to google on compression methods.

 

And I was reading about the usual collection of algorithms, example lzma (7z), and then the article went into discussing image compression, including newer neural network based compression. One of which was Generative Adversarial Network based compression.  OK, here is the funny thing, my tired eyes picked up on a sentence fragment "GAN-based compression can produce higher quality images." 

 

So...here is what I was thinking,  you have lossy compression right, and lossless compression.  And now "additive" compression.

That can now produce higher quality images (than the original)....haha...which of course is not what they meant at all.

 

Damn, my science fiction future was looking sweet. 

 

 

 

 

  • Like 1

Share this post


Link to post
Share on other sites
18 minutes ago, Irgendwer said:

You could create a library with every A8 production

hahah, shortcut ;-)

 

 

graphics could also be converted to fonts and compressed separately the image map and fonts ;-) something that G2F does ... but here you need to compress various data.

 

 

8 minutes ago, Mark2008 said:

So...here is what I was thinking,  you have lossy compression right, and lossless compression.  And now "additive" compression.

watch out ... someday there will be additive CODE algorithms... that will make the game Commando out of the Who Dare Wins II game code :D

  • Like 1

Share this post


Link to post
Share on other sites
10 hours ago, xxl said:

that much memory is needed to decompress this file? : D

 

Actually yes! This build of paq is symmetric in that way, but probably horribly inefficient. It's not really production code.

Share this post


Link to post
Share on other sites

slightly better deflate result:

zopfli --i4000 --deflate conan.gfx

1,598 conan.gfx.deflate

Share this post


Link to post
Share on other sites
48 minutes ago, xxl said:

if anyone is interested in the unShrinkler decompression procedure for Atari 6502, I have shared it here:

 

https://xxl.atari.pl/shrinkler-decompressor/

Nice code. Fun to see how you keep Y at 0 whenever it needs to be 0, and it's 0 again after the mulitply. And there's no ldy #0 anywhere in the code :)

 

Not a geat fan of the @ usage for branches (@- and @-1) and rol @ (and I don't like it that I can't type a single @ on atariage without triggering the username lookup ;) )

 

But as I said, nice code. Looking forward to where this goes in terms of usage in the future.

  • Like 2

Share this post


Link to post
Share on other sites

no, you can easily see that most of these ZP can be moved into code or behind decompressor code.

there should only be 4 bytes left on page zero - dst and tabs


no. the procedure itself initializes what is needed.


buffers yes - in this version 6 pages (in fact 3 but the lower byte tabs is treated as a counter) - the buffer initializes itself.

Share this post


Link to post
Share on other sites

Just having a look at the source. "Jan”, "you", "constant". Macros? What do they do please?

 

Edited by Sheddy

Share this post


Link to post
Share on other sites
7 minutes ago, Sheddy said:

"Jan”, "you", "constant"

??? there are no such words. maybe turn off the automatic www translator maybe it breaks something.

Share this post


Link to post
Share on other sites

Ah yes. Thanks! I hadn't noticed it had translated. That 6502 is much more readable in the original Polish now.

Share this post


Link to post
Share on other sites
On 2/6/2021 at 4:10 PM, xxl said:

Fox joined in optimizing the code which resulted in another speedup.

To lazy to try it out myself, so how much faster is the optimized version in the "Conan-case"? (over 500 frames vs. ?)

Share this post


Link to post
Share on other sites

the first version that was practically launched to see if it works took $22C frames for this picture. My later optmalizations gave $1E3, while Fox optimizations increased the performance to $1DE and have shortened the code (we are talking about the same picture all the time)

 

 

I updated the page because I noticed that 4 labels were missing:

https://xxl.atari.pl/shrinkler-decompressor/

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites
Posted (edited)

another big upgrade, more mature code, this time speeding up by two frames on the test program (image decompression) - bravo Fox. (code size $14E bytes)

Edited by xxl
  • Like 4

Share this post


Link to post
Share on other sites

While this is all interesting in that there are definitely some cases where decompression time doesn't matter, I believe that those circumstances are rare, and that users generally don't like to be kept waiting (too much).

 

From what I can see, nobody has yet mentioned the new-kid-on-the-block ... ZX0.

 

Current testing show this to be as-fast or faster to decompress than APLIB, but with better compression ... so 2x or 3x as fast as Exomizer.

 

lzsa: 1811 (lzsa.exe -f2 -r)

apl: 1655

ZX0: 1625

EXO: 1537 (exomizer raw -E)

  • Like 3

Share this post


Link to post
Share on other sites
Posted (edited)

this ZX0 is pretty cool :)

simple code conversion (if I have some time, I will optimize it) it can be significantly speeded up by replacing the copying procedure, I didn't touch the X register, it will optimize well.

 

dzx0_standard
         lda #$ff
         sta offset
         sta offset+1
         ldy #$00
         sty len
         sty len+1
         lda #$80

; Literal (copy next N bytes from compressed file)
; 0  Elias(length)  byte[1]  byte[2]  ...  byte[N]
dzx0s_literals
         jsr dzx0s_elias
         pha

cop0     lda (compressed_data),y
         sta (decompressing),y
         inw decompressing
         inw compressed_data
         dew len
         lda len
         ora len+1
         bne cop0

         pla
         asl @
         bcs dzx0s_new_offset

; Copy from last offset (repeat N bytes from last offset)
; 0  Elias(length)
         jsr dzx0s_elias
dzx0s_copy
         pha
         lda decompressing
         clc
         adc offset
         sta copysrc
         lda decompressing+1
         adc offset+1
         sta copysrc+1

cop1     lda (copysrc),y
         sta (decompressing),y
         inw decompressing
         inw copysrc
         dew len
         lda len
         ora len+1
         bne cop1

         pla
         asl @
         bcc dzx0s_literals

; Copy from new offset (repeat N bytes from new offset)
; 1  Elias(MSB(offset))  LSB(offset)  Elias(length-1)
dzx0s_new_offset
         jsr dzx0s_elias
         pha
         php
         lda #$00
         sec
         sbc len
         sta offset+1
         bne @+
         plp
         pla
         rts           ; koniec
@        plp
         lda (compressed_data),y
         sta offset
         inw compressed_data
         ror offset+1
         ror offset
         sty len+1
         lda #1
         sta len
         pla
         bcs @+
         jsr dzx0s_elias_backtrack
@        inw len
         jmp dzx0s_copy

dzx0s_elias
         inc len             ; inw ?
dzx0s_elias_loop
         asl @
         bne dzx0s_elias_skip
         lda (compressed_data),y
         inw compressed_data
         rol @
dzx0s_elias_skip
         bcc dzx0s_elias_backtrack
         rts
dzx0s_elias_backtrack
         asl @
         rol len
         rol len+1
         jmp dzx0s_elias_loop

 

 

image.png

zx0-gfx.obx

Edited by xxl
  • Like 2

Share this post


Link to post
Share on other sites
Posted (edited)

 

and a bootloader that loads ZX0 packed binaries

 

I sent a request to Tebe, SuperPacker will receive this functionality ... maybe

 

bootloader-zx0.atr

Edited by xxl
  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)

If I had something to say about to improve functionality of Super Packer, it would be to split the depacker and the compressed data in 2 different segments. So that, the depacker could be used several times as a function. It would save space. I think the only parameters required would be the memory addresses of the compressed data and the target to be decompressed. Such 4 bytes should be inside of the segment of the depacker, being possible to be modified with the program as required. At this moment, for every compressed segment the depacker is duplicated.

 

Edited by tane

Share this post


Link to post
Share on other sites
10 minutes ago, tane said:

At this moment, for every compressed segment the depacker is duplicated.

no if you are using the "SE" version

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...