peteym5 Posted April 5, 2017 Share Posted April 5, 2017 Has anyone optimized "Inflate.asm" so it can inflate into RAM faster or get the routine to use less space? I did a few things with it over the years to see if it can inflate any faster. But this program is outside my normal knowledge dealing with compression. Many times I experimented with something to optimized it, can get garbage loaded into RAM. I did see it was originally inefficient with CPU usage, had many extra JSR calls that could be removed. Anybody else do anything with this program? Quote Link to comment Share on other sites More sharing options...
+CharlieChaplin Posted April 5, 2017 Share Posted April 5, 2017 Hmmmm, whenever the inflate routine was used in an A8 program, it was stated later: "inflate routine by FoX", so you most likely have to ask FoX about it... His real world name is Piotr Fusik and you can find some code and info at Github: https://github.com/pfusik/zlib6502 https://github.com/pfusik/zlib6502/commit/3faede80c34b7a060715873e4834d1fe3acebbfa And no, I am not a programmer, so I never do anything with ASM sources (or any other programming sources)... Quote Link to comment Share on other sites More sharing options...
phaeron Posted April 6, 2017 Share Posted April 6, 2017 (edited) The main reason that decoding Deflate takes so much space is that it uses no less than three Huffman trees per dynamic block: one tree for literal/distance code, a second tree for the length codes, and a third tree to encode the first two (along with run-length encoding!). You can reduce this by making concessions such as hardcoding some or all of the trees and limiting encodings. For instance, restricting distances to 256 bytes would simplify the copy routine. These are difficult to take advantage of without customizing the compressor, however. Before doing that, though, you might want to just check if LZ4 will work; its compression ratio won't be as good, but that may be made up for by a smaller decompressor. Deflate also tends to get poorer compression ratios on small streams due to the relatively large header on each dynamic block, so LZ4 might even get better compression at Atari-scale. Edited April 6, 2017 by phaeron 1 Quote Link to comment Share on other sites More sharing options...
vitoco Posted April 6, 2017 Share Posted April 6, 2017 I've designed a LZ4-like decoder/loader, which requires a specially formatted XEX file. It would be used in old and small cartridges. It is an unfinished project as the menu for a multicart is still under development. The encoder was written in perl language using a LZ4 module, and it builds a LZ4X file as required using a standard XEX. Another perl script builds the .CAR image using those LZ4X pre-encoded files (games). Quote Link to comment Share on other sites More sharing options...
xxl Posted April 6, 2017 Share Posted April 6, 2017 LZ4 for atari: http://xxl.atari.pl/lz4-decompressor/ 2 Quote Link to comment Share on other sites More sharing options...
pirx Posted April 6, 2017 Share Posted April 6, 2017 (edited) You may also look at https://bitbucket.org/magli143/exomizer/wiki/Home-- I found exomizer faster and shorter than deflate/inflate. And compressing a bit better, too. Simply better in every way : ). I guess you know https://github.com/pfusik/zlib6502-- this code is excellent and I would sincerely doubt there is an easy optimisation at all without changing the format as described by phaeron above. (edit, already mentioned by Charlie : ) Edited April 6, 2017 by pirx 1 Quote Link to comment Share on other sites More sharing options...
peteym5 Posted April 6, 2017 Author Share Posted April 6, 2017 Thankyou, I will be doing some bench marking on this modified Inflate routine today. I had ported it to MADS ASSEMBLER and tested it with a game I am working on. I did discover with Deflate/Inflate, the larger the file or block is the better the compression ratio gets, so it is not efficient at blocks under 500 bytes. Things like fonts, large blocks of texts, or sections of the program that need to load into RAM over 2K long is what I had been using it for. As many do know, I had been using deflate on cartridge games for the last 3 years to prevent them from using more expensive components. Get around the need to figure out bank switching and figuring out how to fit stuff into each bank. Makes it easier to debug. Cartridges end up being $40 instead of $50 or $60. I looked into Exomizer when I was figuring out how to fit Venture onto a 16K cartridge, but the routine + compressed block ended up over 16K. Plus I had problems figure out how to get it to decompress to desired memory locations. But I got inflate work, I wanted to phase out the 32K Venture Cartridge as soon as possible, so figuring out Exomizer got shelved. Quote Link to comment Share on other sites More sharing options...
xxl Posted April 6, 2017 Share Posted April 6, 2017 I found exomizer faster and shorter than deflate/inflate. And compressing a bit better, too. Simply better in every way : ). really? source file: gpl3.txt - 35147 bytes exomizer - 12382 bytes + ~256 bytes, decompression time 2.6 sec deflate - 11559 bytes + ~512 bytes, decompression time 3.6 sec LZ4 - 15622 bytes + ~150 bytes, decompression time 1.1 sec 3 Quote Link to comment Share on other sites More sharing options...
Irgendwer Posted April 6, 2017 Share Posted April 6, 2017 deflate - 11559 bytes + ~512 bytes, decompression time 3.6 sec deflate (with zopfli) - 11425 bytes (just for completeness) Quote Link to comment Share on other sites More sharing options...
peteym5 Posted April 6, 2017 Author Share Posted April 6, 2017 I did a benchmark with Venture which uses a 26231 byte binary down into 15778 compressed deflate file. Tested the original routine and decompressed the file into RAM in about 3.5 seconds. I applied this newer version by FoX and it takes about 4 seconds. I am not sure everything I did to modify the original Venture version, but I copyied the getbit/getbyte sections from FoX into my modified version and get it down to about 3.3 seconds. It will take me some time to figure out everything I did and what Fox did over the years. This was something I did 3 years ago and just left it alone because it work. Just recently looked into speeding it up more. Quote Link to comment Share on other sites More sharing options...
peteym5 Posted April 20, 2017 Author Share Posted April 20, 2017 (edited) I am sticking with Inflate/Deflate. Other compression routines do not compress as well, even though they may be faster. The recent version FoX did cut down the size and got it to work a little faster. Most of the games I use it for needs to decompress segments under 2k and done in a split second, so it is unnoticeable. When you need to decompress stuff over 3k, I start noticing delays. I usually turn off the DMA and NMI so it does it faster. This does give your programs another level of protection from hackers, just compress routines, have it inflate, do the checks it needs before executing the main program. It adds an extra step for them to hack your stuff. I have a few segments that are over 8k that take several seconds to inflate. Been looking for ways to improve on that. Edited April 20, 2017 by peteym5 Quote Link to comment Share on other sites More sharing options...
Irgendwer Posted April 20, 2017 Share Posted April 20, 2017 This does give your programs another level of protection from hackers, just compress routines, have it inflate, do the checks it needs before executing the main program. It adds an extra step for them to hack your stuff. Come on, you can do better: First "LZ4" then "deflate" - it just adds an extra step for the hackers. Maybe you find some obfuscating routines too - for the hackers, but be sure you have spend at least the same amount of time to fix bugs and incompatibilities for the CUSTOMERS! 2 Quote Link to comment Share on other sites More sharing options...
+CharlieChaplin Posted April 20, 2017 Share Posted April 20, 2017 Come on, it is 2017, no need to protect any A8 program against hackers. Whatever you do - a hacker will be able to undo. Even if you create the best copy protection ever made on the A8 (better than EA, Synapse, etc.), someone will be able to hack/crack it. As irgendwer already said, better spend your time on gamecode and bugfixing. Here is something to relax for you... 1 Quote Link to comment Share on other sites More sharing options...
_The Doctor__ Posted April 20, 2017 Share Posted April 20, 2017 (edited) This video contains content from Eagle Rock. It is not available in your country.... better in all ways version... What's more is the song is not technically disco which is nice! Edited April 20, 2017 by _The Doctor__ Quote Link to comment Share on other sites More sharing options...
peteym5 Posted April 21, 2017 Author Share Posted April 21, 2017 I hate when people drag these threads off subject. I am looking for ways to the improve Deflate/Inflate decompression times. We all do not like really long delays waiting for a game to load or waiting for the next level. I do not think we can get a better compression ratio than Inflate. The next Secretum Labyrinth will be using compression more intensely. I figured out how to combine several songs into larger RMT files, and get a better compression ratio. About twice as many songs can be put onto a 64K ROM cartridge. That is just one thing I am looking at expanding on with this new game. Quote Link to comment Share on other sites More sharing options...
+Stephen Posted April 21, 2017 Share Posted April 21, 2017 Come on, you can do better: First "LZ4" then "deflate" - it just adds an extra step for the hackers. Maybe you find some obfuscating routines too - for the hackers, but be sure you have spend at least the same amount of time to fix bugs and incompatibilities for the CUSTOMERS! Nah - there are no honest customers in the 8-bit land. Just dirty pirates who steal and spread all the new shit. As much as I want this game, I absolutely will not support anyone with this mentality. I will continue to PayPal donations to people who release their stuff for free, and don't consider people guilty until perceived innocent. 4 Quote Link to comment Share on other sites More sharing options...
_The Doctor__ Posted April 21, 2017 Share Posted April 21, 2017 (edited) Yeah all those titles are on the Warez House BBS anywayz... The dark underbelly of the interwebz on dah telnetz. Those secret decoders are workin' full time! Deflate/Inflate sales optimization rationale ... I think some people don't realize they lose money sometimes. By an estimate of comments here and on other 8 bit forums I think it costs upwards of 27 percent in sales. If I were to take them at their word. By reputation score and consistent posting I tend to think it's a fact that they will avoid the titles.... It is a shame as more money would mean a happier coder and crew. I once refused to sell a truck on principle but that was a one off, not policy. Is that far enough back on topic for you? Old Atari commercial.. do the math! in any event I await the post: Inflate/deflate was optimized by so and so on sucha date the optimization was the code snippet. blah blah The off topic video posting looks to help the topic with some much needed lightening of atmosphere and some good advice, if not just some uplifting ditties... Edited April 21, 2017 by _The Doctor__ Quote Link to comment Share on other sites More sharing options...
Irgendwer Posted April 21, 2017 Share Posted April 21, 2017 (edited) I hate when people drag these threads off subject. I didn't introduce the other flavour. BTW: Compression is evil. If I understand http://atariage.com/forums/topic/260178-tempest-elite-coming-soon-to-atari-8-bit/page-8?do=findComment&comment=3743894 correctly, I'm sure Lance not only likes to support the shipping companies and custom employees but also the Eprom manufactures too (assumed this was no subterfuge). So just go for a bigger and more expensive part (important!) and you have short fetching times and Lance is happy too! I like both of you and the fun you bring in here - really! Edited April 21, 2017 by Irgendwer 4 Quote Link to comment Share on other sites More sharing options...
pirx Posted April 21, 2017 Share Posted April 21, 2017 really? source file: gpl3.txt - 35147 bytes exomizer - 12382 bytes + ~256 bytes, decompression time 2.6 sec deflate - 11559 bytes + ~512 bytes, decompression time 3.6 sec LZ4 - 15622 bytes + ~150 bytes, decompression time 1.1 sec Hi! gpl3.txt is a bad example as it has got much lower entropy than the usual binary file with code / graphics on 8-bitter. Can not prove this right now, but my results for a non-compressed Ataridos binary file were reversed for me and exomizer was faster (no doubts here) and compressed file shorter. Two files in circulation compressed with these 2 algorithms were my 16KiB intro and Amaroute+, in both exomizer won. Quote Link to comment Share on other sites More sharing options...
peteym5 Posted April 21, 2017 Author Share Posted April 21, 2017 (edited) I discovered with different types of files, the compression ratio is different, Text files that decompress directly to the screen have a higher ratio than ML program files. Between the 3 decompression algorithms, I got deflate to work first with my games and that is what I been sticking with. I had a problem getting Exomizer decompressing to the desired memory location and with me needing to get things to work and get these games out, deflate ended up being the algorithm of choice. In most cases, the time it needs to decompress something is not noticeable. The game that takes slightly longer time to boot up is Venture (16K) because deflate is expanding a 20k file into RAM. Delta Space Arena, only needs to expand 4k into RAM. Compression helps keep the costs down with the cartridges. Many more games can be fit into these 16K cartridges that Video61 has in stock that sale for $39.95. Uses 2 8K Eproms. Free shipping within United States. If not, games will need to go onto bigger cartridges that cost more in parts, need to work with bank switching, and more time to put together. Flashcarts, or bigger Eproms pushes the prices higher. Edited April 21, 2017 by peteym5 Quote Link to comment Share on other sites More sharing options...
+CharlieChaplin Posted April 21, 2017 Share Posted April 21, 2017 Hi! gpl3.txt is a bad example as it has got much lower entropy than the usual binary file with code / graphics on 8-bitter. Can not prove this right now, but my results for a non-compressed Ataridos binary file were reversed for me and exomizer was faster (no doubts here) and compressed file shorter. Two files in circulation compressed with these 2 algorithms were my 16KiB intro and Amaroute+, in both exomizer won. Think I created more than 700 TIP animations and in the end compressed them all. Exomizer most of the time had better compression ratio than Inflate/Deflate. (Superpacker by TeBe offers Exomizer, Deflater and LZ4 so I can easily test the different results.) There is also a "Snowman" XEX demo at atarionline available with two versions / two different compression formats, the deflate version is approx. 5% shorter, but therefore takes much longer to de-compress than the other version. Back to compressed games on cart - no-one wants to insert a cart and stare at a black/blank screen for 10 seconds or longer. One may think the A8 has crashed then. Alas, I do have some carts in my collection where this is the case, e.g. the Yoomp cart., it does not show the nice loading animation, like the XEX version did, it only shows a black/blank screen while "loading" and de-compressing. So please, keep the de-compressing short on cart games (5 seconds or less) or show some title/animation while decompressing and not only a black/blank screen... 2 Quote Link to comment Share on other sites More sharing options...
peteym5 Posted April 21, 2017 Author Share Posted April 21, 2017 Think I created more than 700 TIP animations and in the end compressed them all. Exomizer most of the time had better compression ratio than Inflate/Deflate. (Superpacker by TeBe offers Exomizer, Deflater and LZ4 so I can easily test the different results.) There is also a "Snowman" XEX demo at atarionline available with two versions / two different compression formats, the deflate version is approx. 5% shorter, but therefore takes much longer to de-compress than the other version. Back to compressed games on cart - no-one wants to insert a cart and stare at a black/blank screen for 10 seconds or longer. One may think the A8 has crashed then. Alas, I do have some carts in my collection where this is the case, e.g. the Yoomp cart., it does not show the nice loading animation, like the XEX version did, it only shows a black/blank screen while "loading" and de-compressing. So please, keep the de-compressing short on cart games (5 seconds or less) or show some title/animation while decompressing and not only a black/blank screen... This is what I did with the 16K cartridges games after Venture 16K. Delta Space Arena, Laser Blast, etc only has parts of the program/data is compressed and expended to RAM as needed and many cases you only see a blank screen under 0.5 seconds. I got Venture down to 3.5 seconds, looked at ways to get Venture under 3 seconds, either optimize it to the point to where only a small portion needs to be inflated and use the latest version of Inflate. No one has yet panicked about Venture taking too long to boot up. 1 Quote Link to comment Share on other sites More sharing options...
+Stephen Posted April 21, 2017 Share Posted April 21, 2017 This is what I did with the 16K cartridges games after Venture 16K. Delta Space Arena, Laser Blast, etc only has parts of the program/data is compressed and expended to RAM as needed and many cases you only see a blank screen under 0.5 seconds. I got Venture down to 3.5 seconds, looked at ways to get Venture under 3 seconds, either optimize it to the point to where only a small portion needs to be inflated and use the latest version of Inflate. No one has yet panicked about Venture taking too long to boot up. Even the Defender 2000 cart (Jaguar) displays a Lloading screen on powerup. Nobody seemed to mind. Quote Link to comment Share on other sites More sharing options...
_The Doctor__ Posted April 21, 2017 Share Posted April 21, 2017 The ideas and suggestions are to improve or help the products.. unfortunately it is taken as some sort of attack or some such nonsense... A simple "loading" or "saving" has been a courtesy since the dawn of computing.... I do not understand why it is always a conflict when any suggestion is made of any kind. As it is clear no purpose is served I am out of the thread.... and as others have done... I regretfully.... no longer visit or purchase the products or the 'NO SOUP FOR YOU' website.... *ss-hattery by the proprietor must affect those he comes in contact with or some sort of mind meld/mind bending occurs. .... 1 Quote Link to comment Share on other sites More sharing options...
gozar Posted April 22, 2017 Share Posted April 22, 2017 Compression helps keep the costs down with the cartridges. Many more games can be fit into these 16K cartridges that Video61 has in stock that sale for $39.95. Uses 2 8K Eproms. Free shipping within United States. If not, games will need to go onto bigger cartridges that cost more in parts, need to work with bank switching, and more time to put together. Flashcarts, or bigger Eproms pushes the prices higher. If I was doing a cartridge release I'd be looking at piggybacking off the work of Electrotrains and use the STM32F4 as the cartridge: http://atariage.com/forums/topic/257488-racing-the-atari-bus-with-a-stm32f4-microcontroller/?p=3639094 128K and a cost of ~$13. 1 Quote Link to comment Share on other sites More sharing options...
Like us on Facebook
Follow us on Twitter
AtariAge on Instagram
Watch us on YouTube
Chat on Discord
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.