einar
New Members-
Content Count
3 -
Joined
-
Last visited
Content Type
Profiles
Member Map
Forums
Blogs
Gallery
Calendar
Store
Everything posted by einar
-
Data Compression Test on Simple Bitmap 256x192
einar replied to newcoleco's topic in ColecoVision Programming
You are welcome!!! -
Data Compression Test on Simple Bitmap 256x192
einar replied to newcoleco's topic in ColecoVision Programming
No problem! Thanks for the confirmation, I'm glad my ZX7 compressor is not flawed Elias Gamma and Elias Delta are very similar. The difference is, Elias Gamma provides slightly shorter encodings for small numbers, and Elias Delta provides slightly shorter encodings for large numbers. Therefore LZSS compression works better with Elias Delta when expecting a higher percentage of very long matches (such as large files with lots of repetitions using a large sliding compression window), and Elias Gamma works better otherwise. There's a good image in Wikipedia to illustrate this idea: https://commons.wikimedia.org/wiki/File:Fibonacci,_Elias_Gamma,_and_Elias_Delta_encoding_schemes.GIF I'm not sure the paper conclusion you mentioned makes much sense, since this is not supposed to be much related to the matching algorithm used. It depends almost exclusively on the characteristics of the data to be compressed. For 8-bit machines, that typically handles small data blocks (only a few Kb or less), Elias Gamma is certainly the best choice. It seems you are focused on compressing text and full screen graphics only. There's obviously nothing wrong about producing a compressor specialized in certain data types. A specialized compressor would be very useful too. However, if you intend to produce a generic compressor, you better make sure your samples include other kinds of data too. Otherwise you will probably make choices that will benefit compression of certain data types only, but sacrifice others. In match-based compression like LZSS, "greedy matching" (also known as "naive matching") means: at each step, just look for the next longest match and take it. In comparison, "lazy matching" means: at each step, look at next pairs of matches and take the next match from the longest pair. A more detailed explanation is provided here: http://fastcompression.blogspot.com/2010/12/parsing-level-1-lazy-matching.html Everybody assumes it would be too "time consuming" to analyze all possible combinations of matches, in order to achieve the best possible compression. For this reason, existing algorithms only look at 1 (greedy) or 2 (lazy) matches ahead. But they are wrong. Just look at ZX7... Combining LZ and RLE is a good idea. PuCrunch is also based on this idea. Anyway your results are very promising. I will be looking forward to your final version! I think the most useful advice I can give you is: If you concentrate on optimizing your algorithm against a few sample files, you will obtain the best possible algorithm specialized in compressing these sample files only, instead of the best possible algorithm for most typical files. The only way to avoid this "trap" is, from time to time, add more files to your test sample in order to validate your conclusions. -
Data Compression Test on Simple Bitmap 256x192
einar replied to newcoleco's topic in ColecoVision Programming
Hi, My name is Einar Saukas, I'm the original author of ZX7. I'm glad to know people have been using it here, thanks for the interest! However I'm afraid there's some misunderstanding I would like to correct: This information is wrong. The ZX7 compression tool is "optimal", which means it produces the best compression results possible for this format. The documentation file explicitly says it. If you read anywhere a comment that suggested otherwise, please let me know so I can post a clarification there too. The compression tool runs very fast because it implements a faster algorithm I developed myself, but it doesn't sacrifice compression ratio to achieve this performance. I'm fairly confident my ZX7 compressor is optimal, thus it should never be possible to produce an even smaller result for any file using the same format. However I recognize I'm not infallible, there's always a chance I made a mistake somewhere. Could you please provide one example where you believe you got a better result? If I really made a mistake, I would very much appreciate the chance to learn about it! Sounds interesting! I'm curious about your format also. Where can I find it? When I designed ZX7, I was aiming for the best tradeoff between compression ratio on most typical cases, decompression speed and size, while restricting myself to "pure" LZ77/LZSS. I'm aware that different file formats (consequently different algorithm strategies) could potentially provide better results in one of these characteristics by sacrificing others, and it's always interesting to see different algorithms and the choices they made.
