rensoup

I wasn't asking for myself, just suggesting that a tutorial with a working example is even better (just something simple like a color change) I've seen some multiplexor demos, I was complaining about changing multiple sprite positions/color every scanline not being practical One thing I'm not sure of and I see you don't mention it, is checking for the source of the NMI when one occurs. I made the assumption that it can be omitted but I'm not sure how safe it is. Another trick to get some cycles back when chaining DLIs is to only update the low byte of the NMI vector if you're not crossing a page... that's pretty obvious indeed but that's my modest contribution to this thread

Nice of you doing this tutorial, a fully working sample would be great! I actually found code posted on AA for doing this (Thanks to Popmilo I think!) and it really helped. My brief experience with them: -I was naively hoping I could get around the sprite limitations by changing their positions/sizes/colors every scanline. As it turns out, it's practically impossible. Even though you may have the CPU time, the changes would occur all over the scanline and cause all sorts of glitches. -I think the concept was a little ahead of its time. It would have been ok with a 6502 at twice the clock speed. It works great on an ST or Amiga. -I woud like to know why on earth they didn't time DLIs to actually start towards the end of the scanline. From my tests, they start in the middle of it... this means you have to waste a lot of cycles with a costly WSYNC to hide colors changes. Those colour changes every scanline which the Atari is famous for can't actually be done with DLIs without wasting most of your CPU cycles... It's crazy.

lzs16: 3.7KB !! Can't wait for the next release

Well yeah I somehow missed it, sounds really nice (and cpu intensive)!

Tried it on 7 gates of Jambala (110KB SAP). Lzs8: 30KB Lzs12: 8.5KB The new version can be a tiny bit slower but still several times faster than the original player! Guess it's an all around solution now

I didn't know Sid2Gumby, was only aware of the latest iteration of your AtariSid... Honestly, S2G feels a little dated, especially stacked against AS5 . Wish I could SAPR those instead... but yeah at 15khz that's not going to happen (although only the first channel seems to vary much during a frame) I don't know the tech behind AS5, nor do I know anything about Pokey . But just out of curiosity, I think I read you were doing software mixing, does that mean a Sid voice is not mapped to a Pokey voice ? although according to the PMG visualizer that seems to be the case.

I tried on a 8KB RLE compressed song and all channels but one got bigger. The one that got smaller shrunk by 500 bytes! But I couldn't be bothered. Again I'm thinking I'd get better results by modifying the RLE format but I'd be trying to catch up with dmsc so not much point. Perhaps larger buffer sizes in dmsc's lzss would yield good gains too? It's pretty usable for any tune under 2 minutes right now. Just curious about your Sid player, are you feeding pokey at a higher frequency than 50hz ? (I'm guessing the sprites shapes in your player reflect that)

About looping, I did -some- of those things you mentioned, I'll give it another try. Thanks! I've been wondering about SAP->SAPR myself... Only found ASAP so far: http://asap.sourceforge.net/ There's a command line tool included which I haven't tried yet. On Win, there's a player called WASAP included in the package. If you load a song and go to info, you can save a XEX and SAPR it from Altirra. Best I found was to get an RMT and assemble it, then SAPR it in Altirra and KIL on song loop to get a perfect loop

Gave it a shot... the player is quite a bit faster indeed. Litterally 4-5 times faster than the original RMT sometimes! Only issue is that I can't seem to get it to loop 100% right with the new version... not excluding that I've done something wrong of course... zip contains just obxs for all versions for a quick perf compare. shadowsobx.zip

For ./lzss < test1.sap > test1.lzs, it's shadows.zip which I posted above. my sapRLE: +-8KB dmsc: 6.6KB Are we going to end up with 3 solutions ?

OOoops Sorry, I'd ORGd the tune at $1000 and forgot that your player is ORGd at $2000, so the player ended up playing itself! Verified to work again, added looping, looping perfectly!

actually, I'm still having an issue further in the song, it's still before the loop point I believe. Maybe another redirection issue ? I'm attaching the obj and the converted file lzsfail.zip

Yes I'm under Windows, so your fix worked perfectly. The files are a bit bigger now but still slightly smaller than my RLE. Thanks again! I'm still baffled that nobody has done this before Guess I won't use my RLE . Your code is much smaller, speed is the same, files can be a little smaller or quite a bit smaller, and there aren't any silly restrictions (I have to ORG the compressed data, and have to set manually the song duration in the code for looping. ) I'm just gonna go ahead and post it anyway so you can have a laugh If you look at some of the compressed channels files, the compression is outrageously bad. I could probably implement RLE on duplicate sequences and save a good amount of space but compared to your solution, it wouldn't be very elegant code. sapRLE.zip

Yes, very clear now, I thought you meant standard RLE. GREAT! :thumbsup: Well... almost. I've tried it on a bunch of SAPs and they all fail to decrunch/play at some point... not sure if I've done anything wrong ? I've attached an example that failed. (I did "lzss <shadows.sap >test.lzs") So I got my modified RLE to work and for a 35KB raw, RLE would get it down to 12.5KB, my modified RLE did 9KB, your LZS did 8.5KB. But your LZS did better on other files. The decode time on the 6502 was similar, about 10 scanlines. There are others things that I could do that might improve the compression quite a bit but I'm ok with what I got now. And if your LZS works I'll probably use that instead! Your method is much cleaner, my setup code is a lot more messy and I have to expand the files to make them multiples of 9 when packing. Sure a better compression ratio would be great but as it is, it makes it possible using some of those costly RMTs into actual productions. PS:People have suggested delta encoding, I gave it a quick shot but ended up with results slightly worse. shadows.zip

Yes, I wasn't very clear on that, speed isn't essential in this case (it is for decrunching sprites). I managed to get my RLE stuff working and the code is horrible... way too much setup code. dmsc' solution is much cleaner.

Well I got curious, so I took my 9 deinterleaved raw SAPR channels and compressed them with LZSS v0.01. and the total was about 4KB vs 2KB for LZ4 best setting. That's just one very tiny set of data and perhaps that wasn't the best LZSS compressor... Well thanks for that! The obvious reason for that would be that the compression ratio may be somewhat worse, I guess ? Well, call me lazy but I've not used C in years (I'm all C# these days) and I don't even think I installed it in my VS.

Well that's some weird stuff Not sure it's worth it, let me pm you to explain...

Damn and I was hoping for an existing solution

Ok that was confusing... you're saying that instead of of using the regular LZ4 encoder, I could use the LZ4 api to set the LZ4 dictionary to come from the previously compressed source data, instead of the decompressed destination ? something like: For every new chunk of 9 bytes to compress: -Set dictionary to be compressed source data, so the dictionary grows a bit more for every 9 byte chunk which in theory improves compression every time -Compress 9 bytes using new dictionary. ? Yes that bit I know, but these are offset in the decompressed data but from you're saying above I could change that to use offsets in the compressed source ?

Ooops you're right. dmsc's version seems a lot faster though.

found this post: https://forums.nesdev.com/viewtopic.php?f=2&t=13671 ouch...

Thanks, I found a 65c816 decoder for it ( https://forums.nesdev.com/viewtopic.php?f=12&t=5771) but its author says it requires 4KB of mem... that unfortunately seems like a no go for me! Well that's why I was hoping for a modified RLE, because the non repeated bytes are already stored decompressed in the source. So if some sequences are repeated, you could just store an offset and length. I'm just surprised that nobody has done that. I actually gave it a shot yesterday... it was a lot more annoying than I thought but it seems to work. The gains vary from nothing to about 25% compared to regular RLE and I get the feeling there's more that can be done. At 300-600 bytes/frame I would agree but dmsc's LZ4 decoder benchmark seems promising though.

Indeed, not what I want to do at all though... sorry for the confusion Yes unfortunately for SAP-R, the window has to be 128-256 bytes for any decent compression, 128 *9 *2 =~2.3KB just for buffers, you're starting to lose the benefit of the compression. I get the feeling that the code would be clunky too. A cheaper compressor might do just as well.

Ok, so I guess that was a little confusing. I mixed 2 use cases into one question. My first use case was simple: Like pirx said a decent streaming compression scheme would be perfect for SAP-R which exactly what I was trying to do! lz4 deinterleaved Pokey data compresses pretty well... so I modified XXL's lz4 decompression code to only output 9 bytes per frame only to realize that it didn't work because it was trying to copy data that I'd already discarded (yeah I know very little about compression ) I also realized that it was taking about 10 scanlines to output 9 bytes! I had another use case in mind (compressing sprites) but at 300 bytes/frame, it just wouldn't cut it either. With the code posted before, I have hope for this one.

well 15.3KB/ 55 =280 bytes per frame which is what I was seeing as well... Sure it's faster than exomizer/deflate but they're at the far end of the spectrum when it comes to speed. I was looking for something between lz4 & rle in terms of speed and compression ratio. Looks like I've got 2 possibilities with dmsc & irgendwer at least when it's ok for self referencing. btw I tried compressing data with your rle implementation that I found in MADS, it seems to perform a litte worse than this one sometimes (files can be 1-10% bigger): https://csdb.dk/release/?id=34685