Jump to content
rensoup

Any compressor between rle and lz4 ?

Recommended Posts

Posted (edited)

BTW you can also record the converted Pokey values and stream those. Perhaps you can just record a sid2gumby track with Altirra. Don't know if it supports stereo SAP-R.

 

Edit: BTW2: this does not necessarily mean way more streams (16 or 18). AUDCTL is constant and one sid voice is mirrored on the second pokey. Also, IIRC, the AUDC values of one half of the 16-bit channels are constant. But even if that's not the case, you end up with 6 AUDF/AUDC pairs, hence 12 streams of importance. Just need two extra stores for the mirrored sid voice.

Edited by ivop

Share this post


Link to post
Share on other sites

BTW you can also record the converted Pokey values and stream those. Perhaps you can just record a sid2gumby track with Altirra. Don't know if it supports stereo SAP-R.

 

Edit: BTW2: this does not necessarily mean way more streams (16 or 18). AUDCTL is constant and one sid voice is mirrored on the second pokey. Also, IIRC, the AUDC values of one half of the 16-bit channels are constant. But even if that's not the case, you end up with 6 AUDF/AUDC pairs, hence 12 streams of importance. Just need two extra stores for the mirrored sid voice.

 

I didn't know Sid2Gumby, was only aware of the latest iteration of your AtariSid... Honestly, S2G feels a little dated, especially stacked against AS5 :). Wish I could SAPR those instead... but yeah at 15khz that's not going to happen (although only the first channel seems to vary much during a frame)

I don't know the tech behind AS5, nor do I know anything about Pokey :grin:. But just out of curiosity, I think I read you were doing software mixing, does that mean a Sid voice is not mapped to a Pokey voice ? although according to the PMG visualizer that seems to be the case.

Share this post


Link to post
Share on other sites

IDK as was just listening to a song in another thread that captured the angry bees perfectly... I think emkay has the bees all trained and spinning about in pokey just like they do in sid...

Share this post


Link to post
Share on other sites

Hi all!

 

Perhaps larger buffer sizes in dmsc's lzss would yield good gains too? It's pretty usable for any tune under 2 minutes right now.

Interesting that stream #3 actually increases in size (probably the drum channel), but stream #1 and #5 decrease in size, with an overall saving of an extra 85 bytes. But it would need an extra operation during decompression.

I just implemented 12 bit matches in the LZSS coder/decoder, this allows using larger window sizes, and produces a big gain in compression ratio, compare:

 

SHADOW.SAP:

 

8 bit match, 16 bytes window: max offset = 16, max len = 17, match bits = 8, ratio: 5675 / 42759 = 13.27%

12 bit match, 128 bytes window: max offset = 128, max len = 33, match bits = 12, ratio: 2201 / 42759 = 5.15%

 

3D_RMT.SAP (from RMT distribution, "3D, Atari version by raster/c.p.u. 2009"):

 

8 bit match, 16 bytes window: max offset = 16, max len = 17, match bits = 8, ratio: 15493 / 53568 = 28.92%

12 bit match, 128 bytes window: max offset = 128, max len = 33, match bits = 12, ratio: 4800 / 53568 = 8.96%

 

 

As the player is only 180 bytes, in the case of the "3D_RMT" sample, the player now is smaller than the original RMT player, 4980 v/s 5259 bytes.

 

The new "lzss" program accepts options, use "-8" for original 8 bit matches, "-2" for new 12 bit matches, or any other combination to try other match window sizes.

lzss-sap-20190527.zip

shadows-12.xex

3d_rmt-12.xex

3d_rmt.xex

  • Like 6

Share this post


Link to post
Share on other sites
Posted (edited)

 

I didn't know Sid2Gumby, was only aware of the latest iteration of your AtariSid... Honestly, S2G feels a little dated, especially stacked against AS5 :). Wish I could SAPR those instead... but yeah at 15khz that's not going to happen (although only the first channel seems to vary much during a frame)

I don't know the tech behind AS5, nor do I know anything about Pokey :grin:. But just out of curiosity, I think I read you were doing software mixing, does that mean a Sid voice is not mapped to a Pokey voice ? although according to the PMG visualizer that seems to be the case.

 

https://github.com/ivop/atarisid

 

It seems you have missed AtariSid 6? The binaries are in the xex directory.

 

You are right that the PMG's do not vary much, but that's because they only tell the volume, at 50Hz :) The width is not based on the samples. There simply wasn't enough cpu time to show some sort of osciloscope when I managed to switch from 7.8kHz to 15.6kHz.

 

Atarisid uses Pokey's first channel as a 15.6kHz timer. Each time it fires, it writes to pokey channels 2, 3 and 4. The whole emulation is a sort of a soft-synth.

Edited by ivop
  • Like 2

Share this post


Link to post
Share on other sites

Hi all!

 

 

 

I just implemented 12 bit matches in the LZSS coder/decoder, this allows using larger window sizes, and produces a big gain in compression ratio, compare:

 

 

 

Tried it on 7 gates of Jambala (110KB SAP).

 

Lzs8: 30KB

Lzs12: 8.5KB

 

The new version can be a tiny bit slower but still several times faster than the original player!

 

Guess it's an all around solution now :-D

  • Like 1

Share this post


Link to post
Share on other sites

Hi!

 

Tried it on 7 gates of Jambala (110KB SAP).

 

Lzs8: 30KB

Lzs12: 8.5KB

 

The new version can be a tiny bit slower but still several times faster than the original player!

 

Guess it's an all around solution now :-D

Main difference is in the RAM usage, as the 12 bit version uses 128 bytes for each stream.

 

I implemented a third version, with 16 bit matches. Using 8 bits for the window size (so 256 bytes for each stream), the compression is much better, tested it with 4 samples (attached original and compressed files):

 

---- shadows.sap -----
 max offset= 16,  max len= 17,  match bits= 8,  ratio:  5675 / 42759 = 13.27%
 max offset= 128, max len= 33,  match bits= 12, ratio:  2201 / 42759 =  5.15%
 max offset= 256, max len= 256, match bits= 16, ratio:  1103 / 42759 =  2.58%

---- 3d_rmt.rsap -----
 max offset= 16,  max len= 17,  match bits= 8,  ratio: 15493 / 53568 = 28.92%
 max offset= 128, max len= 33,  match bits= 12, ratio:  4800 / 53568 =  8.96%
 max offset= 256, max len= 256, match bits= 16, ratio:  3536 / 53568 =  6.60%

---- 4tk35.rsap ------
 max offset= 16,  max len= 17,  match bits= 8,  ratio: 27051 / 106587 = 25.38%
 max offset= 128, max len= 33,  match bits= 12, ratio:  9442 / 106587 =  8.86%
 max offset= 256, max len= 256, match bits= 16, ratio:  6342 / 106587 =  5.95%

---- aurora.rsap -----
 max offset= 16,  max len= 17,  match bits= 8,  ratio: 38741 / 114048 = 33.97%
 max offset= 128, max len= 33,  match bits= 12, ratio: 14495 / 114048 = 12.71%
 max offset= 256, max len= 256, match bits= 16, ratio: 11903 / 114048 = 10.44%
As you see, now "shadow.sap" is a little more than 1kB. My samples are from the RMT128 distribution, converted to SAP type-R.

 

Have Fun!

lzss-sap-20190529.zip

3d_rmt-16.xex

shadows-16.xex

aurora-16.xex

4tk35-16.xex

samples.zip

  • Like 7

Share this post


Link to post
Share on other sites

great tool DMSC

 

playlzs16.asm (aurora.lz16) plays wrong

Share this post


Link to post
Share on other sites

Hi!

 

great tool DMSC

 

playlzs16.asm (aurora.lz16) plays wrong

Thanks.

 

I'm not near the PC now, but you must use te following command line:

 

  lzss -6 input.rsap test.lz12
[code]

This is the same as:

[code]
  lzss -b 16 -o 8 -m 1 input.rsap test.lz12

Share this post


Link to post
Share on other sites

IDK as was just listening to a song in another thread that captured the angry bees perfectly... I think emkay has the bees all trained and spinning about in pokey just like they do in sid...

 

Hehe.

 

To wich one do you refer exactly?

Share this post


Link to post
Share on other sites

Another idea: turn the compressed data+player into a SAP file again, but this time Type B :D

 

This would allow existing SAP players without SAP-R support to play these songs (i.e. ALL of them, except for Altirra, but that's not strictly a SAP player).

Share this post


Link to post
Share on other sites

Hi!

 

 

Main difference is in the RAM usage, as the 12 bit version uses 128 bytes for each stream.

 

I implemented a third version, with 16 bit matches. Using 8 bits for the window size (so 256 bytes for each stream), the compression is much better, tested it with 4 samples (attached original and compressed files):

 

 

 

Tried it on 7 gates of Jambala (110KB SAP).

 

Lzs8: 30KB

Lzs12: 8.5KB

 

lzs16: 3.7KB !!

 

Can't wait for the next release ;)

  • Like 2

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...