FileLoader "!" - for tape in 1 block

[_The Doctor__]

It's always good to see everything taken to very best it can be, so yes we care!

Always striving for more makes the great Atari world the best. All because people do just this!

[baktra]

I  have a temptation to take the NHP loader source code from github, cannibalize it and assimilate into TS.

 

1. It has all what a good cassette binary loader should have - longer blocks, proper support for INIT segments doing MOTOR OFF/ON and it is not located in RAM under ROM.

2. It has something that is not that much needed - custom screen and messages and countdown. Countdown is lovely as an option, though.

 

@xxl

Any ides for the name of the 1-block loader? The 'X' loader for example?

 

 

[xxl]

31 minutes ago, baktra said:

xxl

Any ides for the name of the 1-block loader? The 'X' loader for example?

x is ok but wait a moment because I have newer versions of loaders but I'm waiting for Tebe to publish SP (he said it would be soon)

 

31 minutes ago, baktra said:

longer blocks

this is simple to use but not necessarily an advantage ... if we are considering error correction it may even be a disadvantage ...

 

but it is true that it would be good to be able save binaries with a record of any length - there were copy programs where individual blocks had different lengths ? 

Edited April 12, 2021 by xxl

[baktra]

7 hours ago, xxl said:

x is ok but wait a moment because I have newer versions of loaders but I'm waiting for Tebe to publish SP (he said it would be soon)

 

this is simple to use but not necessarily an advantage ... if we are considering error correction it may even be a disadvantage ...

 

but it is true that it would be good to be able save binaries with a record of any length - there were copy programs where individual blocks had different lengths ? 

Of course. I will wait with official release for the finalized loader. Git is something else. And the loader is finalized when you say so :-), not sooner.

[Wilheim]

On 4/12/2021 at 5:45 AM, baktra said:

I  have a temptation to take the NHP loader source code from github, cannibalize it and assimilate into TS.

 

1. It has all what a good cassette binary loader should have - longer blocks, proper support for INIT segments doing MOTOR OFF/ON and it is not located in RAM under ROM.

2. It has something that is not that much needed - custom screen and messages and countdown. Countdown is lovely as an option, though.

 

@xxl

Any ides for the name of the 1-block loader? The 'X' loader for example?

 

 

Have you considered to add “Turbo Tennis” support on sturgeon? It’s restricted, but it’s nice to have some game to play during the load.

[baktra]

9 hours ago, Wilheim said:

Have you considered to add “Turbo Tennis” support on sturgeon? It’s restricted, but it’s nice to have some game to play during the load.

Only as an 'if there is really time to spare' option. The restrictions make the loader to be a misfit for the product.

I was thinking how to incorporate even such loaders without making mess of the Standard Plugin. The best way would seem to be a standalone plugin (Standard Extreme, for example) that would allow such conversions.

[baktra]

That's what happens, when one is blind.

LZ4 Decompression in Free Pascal is already here for many months. I will take it and rewrite to Java, so Turgen will understand the hybrid binary files. Duh. 

[xxl]

remember ID: 00 = LZ4, 01 = aPLib, 02 = ZX0

 

I included a virtual decompression procedure and identification of such binaries with Chkxex - the sources are available in Pascal.

 

http://madteam.atari8.info/uzytki/chkxex.7z

 

I am wondering about this error check, actually it is very simple but I would doi it like this: frame color red = error - let the user know with the frame color that the block has a checksum error. (I didn't change Antic's prorgam, didn't write anything to the screen's or to the E:ditor.)
the following questions also arise: should the record counter be extended to two bytes? because with one byte we have max. 32K binary. so either one byte of the counter and records about twice as long, or two bytes per record counter ...

[baktra]

I have committed initial support for Hybrid Binary Load (HBLs) files for Turgen System. 

 

What works?

1. The binary load file parser tolerates files with compressed segments - LZ4 only, for now. If I can be sent a zip file with sample HBLs, I will be happy. I can create my own, using Super Packer.

2. The File Statistics Dialog, Tape Image Extractor and Automatic Silence List generators accept HBLs always. That is desired. Compressed segments are reported as CDATA. 

 

How it works?

The parser just wanders through the compressed data and determines where it ends (to see where the next segment begins). No real decompression takes place. It is not needed anyway.

 

What requires additional efforts.

The Wizard and Batch Processor (BP) accept HBLs when creating playlist items (even though the given plugin/convertor has no support for them).

HBLs are rejected later during attempts to create output. This will change in the future and the Wizard and BP will check for HBL support first.

 

Should you celebrate?

Not yet.

1. The only plugin that will truly support HBLs will be the Standard plugin, through the xxl's loader. I don't think the PWM-based turbo system will ever receive their loaders, but one never knows.

2. Support for HBLs hasn't been thoroughly tested. If you are not adventurous, wait for official TS release.

Edited April 15, 2021 by baktra
Image

[baktra]

I can report vast improvements in processing of the Hybrid binary load files.

 

1. The Wizard for Files and Batch Processor reject HBLs politely when not supported by the underlying conversions

2. When a specific conversion doesn't support HBL, you will get a good message "Compressed segments not supported by the selected function" instead of misleading "Segment with negative size found"

3. The "Check loader" function provides better messaging in general. It had to be touched because of the compressed segments anyway.

 

A weekend is coming, so I will perform some tests with real hardware and report back.

[baktra]

The latest commit expands support for the ZX0 compression in the HBLs. So now I have LZ4 SE and ZX0. Reminds me of the good times at university, when I had to go through the X36KOD course. Passed in 1st enrollment, 2nd exam try, by the way.   APLIB is yet one to go, so I have to steal the decompression routine and rewrite to Java.

 

This shows one unfortunate weakness of the HBLs - to just skip a compressed segment, you have apply the decompression algorithm - at least to pass through. But the algorithms are available, so it is something I can live with.

 

The Conversion of HBLs is available from the user interface. Pilot signal is properly elongated after blocks that hold INIT segments. Unsupported compression type is reported, when encountered.

 

 

The complexities of the HBLs made me start thinking about the future of my xex2cas utility as it is. The idea of duplicating the code in XEX2CAS is nothing I would be fond of. So, I am contemplating a radical idea. Scrap xex2cas as a standalone project, but create a new command line "face" for Turgen System. The new face would be shipped as xex2cas.exe, accepting the same command line parameters as the deceased xex2cas, but calling TS under the cover. Yes, it will require Java, but Java is well-established, mature and available. And except the platform-specific wrappers, all with one .jar file for major PC operating systems.

 

Edited April 17, 2021 by baktra

[baktra]

Two observations:

 

1. Both X loader and X Hybrid loader do need at least 2 seconds of pilot tone after blocks that hold INIT segments. From TS perspective, if you do not specify a silence list, 2.5 seconds pilot tone is added automatically, which should be enough. Only be careful when specifying the silence list explicitly. TSCBL needs these two seconds too, so it is expected behavior.

 

2. I am waiting for new Super Packer and X Hybrid loader. Whatever I pack with LZ4 SE with Super Packer 6.8, is not loaded by the X Hybrid loader. I will re-view my process, though.

 

In the meantime, I will work on the remaining compression type and test with real HW using these MP3s.
Landscape_X_Compressed.mp3 - Hybrid binary file loaded by the X hybrid loader.
Swapz_X.mp3 - Standard binary file loaded by the 1-block X loader.

 

 

[baktra]

I am happy to report that the latest commit has the aPlib decompression working. This time, I copy pasted the C source code to a .java file and kept roasting until a Java appeared. A credit to the author of the C library, who encapsulated the decompression state in a struct.

[xxl]

great, probably it will be best to prepare 3 variants of the loader, each with a different decompressor (LZ4, aPLib, ZX0) and possibly also one that integrates all of them, but then the loader will be the size of this loader "!" :D

[baktra]

12 hours ago, xxl said:

great, probably it will be best to prepare 3 variants of the loader, each with a different decompressor (LZ4, aPLib, ZX0) and possibly also one that integrates all of them, but then the loader will be the size of this loader "!" :D

You were reading my mind. Turgen and XEX2CAS can detect the compression types and choose one of these loaders automatically.

[baktra]

In the meantime (waiting for loaders and Super Packer), I made significant progress on the xex2cas face of Turgen System.

Some of that work was a background work - better decoupling of the signal generators from the GUI part, some was on the command line and distribution. I already have working xex2cas.exe that calls TS behind the scenes. Its is not finished, of course, but simple commands like xex2cas file.xex file.cas already work as expected. 

[baktra]

This is basically a bump for the HBL loaders.

 

I did the background work in Turgen System, both for HBLs and the command line interface (which I renamed to gencas, so it is distinguishable from xex2cas).

I've also found some bugs in TS (not related to HBLs), so I have released 8.8.2, which fixes the bugs. Code for HBLs and gencas is there, but it is not accessible by any user interface.

[jhusak]

XOBLoader (xxl One Block Loader)

X3CLoader (xxl 3 Compressed Loader, Xec loader)

[xxl]

I'm also waiting for Tebe to add ZX0 support to his SuperPackra - without that the user would have to manually create such files and it's not easy: /

[xxl]

@tebe