FujiNet Config loading faster

[pirx]

beautiful development, kudos!

[apc]

Config loader update!

 

After exercising with all the helper scripts and tools, here is an image which contains loader & FujiNet CONFIG (and config tools):

 

autorun-zx0.atr

 

As the name indicates the new ZX0 compression is used. With ZX0 the compressed CONFIG is smaller and it loads a bit quicker compared to LZ4 variant.

ZX0 decompression routine was embedded into boot loader. Both Config loader and CONFIG are decompressed while being read from disk.

As it was before, HISIO is enabled as soon as possible to speedup SIO transfers and CONFIG is loaded with banner and growing progress bar.

 

Build process was automated, "make dist" should do all the hard work. For this purpose I prepared Python script which acts as very simplified SuperPacker (and other script to prepare somehow relocatable code and other script to update final ATR, and ... it was interesting to get all this working together)

 

As a bonus, once the CONFIG is compressed with ZX0 the file segments are "decorated" to make the result Atari DOS compatible/loadable again. However DOS cannot decompress file segments (there are some exceptions ) so decompression routine is bundled and called prior passing the control to CONFIG. I.e. Same CONFIG file can be loaded by ZX0 capable loader (during boot) with inline decompression and it can be loaded as a regular DOS program later from disk.

 

Credits: @xxl for ZX0 decompression port to 6502 and boot loader @HiassofT for high speed SIO @bocianu for FujiNet banner @pirx for compression ideas  🙂

 

Source code: https://github.com/a8jan/fujinet-config-loader - branch "zx0" (If everything works fine, will be glad to move into FN repos)

 

Jan

 

[tschak909]

how do I test?

-Thom

 

[mozzwald]

11 minutes ago, tschak909 said:

how do I test?

-Thom

 

replaced autorun.atr with the new one above in fujinet-platformio/data/ then upload filesystem

 

soooper fast, me likes  

[apc]

23 minutes ago, tschak909 said:

how do I test?

-Thom

 

- mount as D1: and boot - CONFIG should load

- mount as D2: with some DOS disk in D1: boot DOS and load CONFIG - CONFIG should load

- replace autorun.atr with autorun-zx0.atr (rename to autorun.atr), build firmware image, test with firmware

@mozzwald can you test if the loader build process works (git clone ... make dist) on your side? then we can test if build process works on WSL .. would be nice

Edited April 20, 2021 by apc

[mozzwald]

3 minutes ago, apc said:

- mount as D1: and boot - CONFIG should load

- mount as D2: with some DOS disk in D1: boot DOS and load CONFIG - CONFIG should load

- replace autorun.atr with autorun-zx0.atr (rename to autorun.atr), build firmware image, test with firmware

@mozzwald can you test if the loader build process works (git clone ... make dist) on your side?

Building relocatable ZX0 decompressor
../tools/relgen.py zx0unpack-1000-f.obj zx0unpack-1201-f.obj ../tools/pack/a8/zx0unpack.obj
Traceback (most recent call last):
  File "../tools/relgen.py", line 208, in <module>
    main()
  File "../tools/relgen.py", line 142, in main
    with open(fnout, 'wb') as fout:
FileNotFoundError: [Errno 2] No such file or directory: '../tools/pack/a8/zx0unpack.obj'
Makefile:60: recipe for target '../tools/pack/a8/zx0unpack.obj' failed

this is as far as I got

[apc]

4 minutes ago, mozzwald said:

Building relocatable ZX0 decompressor
../tools/relgen.py zx0unpack-1000-f.obj zx0unpack-1201-f.obj ../tools/pack/a8/zx0unpack.obj
Traceback (most recent call last):
  File "../tools/relgen.py", line 208, in <module>
    main()
  File "../tools/relgen.py", line 142, in main
    with open(fnout, 'wb') as fout:
FileNotFoundError: [Errno 2] No such file or directory: '../tools/pack/a8/zx0unpack.obj'
Makefile:60: recipe for target '../tools/pack/a8/zx0unpack.obj' failed

this is as far as I got

ok, give me a minute, i guess, git does not store empty directory ...

[apc]

workaround: mkdir tools/pack/a8

going to put some placeholder file there to keep the a8 dir in git

[mozzwald]

3 minutes ago, apc said:

workaround: mkdir tools/pack/a8

going to put some placeholder file there to keep the a8 dir in git

indeed, adding the dir makes it work

[apc]

Just now, mozzwald said:

indeed, adding the dir makes it work

good, should be fixed in repo

[xxl]

11 hours ago, apc said:

Config loader update!

 

After exercising with all the helper scripts and tools, here is an image which contains loader & FujiNet CONFIG (and config tools):

 

autorun-zx0.atr 90.02 kB · 2 downloads

 

As the name indicates the new ZX0 compression is used. With ZX0 the compressed CONFIG is smaller and it loads a bit quicker compared to LZ4 variant.

ZX0 decompression routine was embedded into boot loader. Both Config loader and CONFIG are decompressed while being read from disk.

As it was before, HISIO is enabled as soon as possible to speedup SIO transfers and CONFIG is loaded with banner and growing progress bar.

 

Build process was automated, "make dist" should do all the hard work. For this purpose I prepared Python script which acts as very simplified SuperPacker (and other script to prepare somehow relocatable code and other script to update final ATR, and ... it was interesting to get all this working together)

 

As a bonus, once the CONFIG is compressed with ZX0 the file segments are "decorated" to make the result Atari DOS compatible/loadable again. However DOS cannot decompress file segments (there are some exceptions ) so decompression routine is bundled and called prior passing the control to CONFIG. I.e. Same CONFIG file can be loaded by ZX0 capable loader (during boot) with inline decompression and it can be loaded as a regular DOS program later from disk.

 

Credits: @xxl for ZX0 decompression port to 6502 and boot loader @HiassofT for high speed SIO @bocianu for FujiNet banner @pirx for compression ideas  🙂

 

Source code: https://github.com/a8jan/fujinet-config-loader - branch "zx0" (If everything works fine, will be glad to move into FN repos)

 

Jan

 

video or didn't happen

[tschak909]

My RetroTink actually takes a few moments to sync up to my Atari's display, and by that time, the status bar is already gone and status loading. (Of course, I am running HSIO)

 

-Thom

[tschak909]

Normal 1X Speed:

 

 

High Speed:

 

 

[apc]

On 4/21/2021 at 5:25 PM, tschak909 said:

Normal 1X Speed:

High Speed:

I do not see much difference. It is so fast capture device cannot get it

 

On 4/21/2021 at 11:30 AM, xxl said:

video or didn't happen

It must be true. It's on YouTube!

Config loader presented in a format which young generation can understand.

 

🙂 Jan

Edited April 22, 2021 by apc

[Mazzspeed]

On 4/23/2021 at 9:41 AM, apc said:

I do not see much difference. It is so fast capture device cannot get it

 

It must be true. It's on YouTube!

Config loader presented in a format which young generation can understand.

 

🙂 Jan

Oh wow, that is a significant improvement!

 

Is this part of Fujinet-PC yet? Don't crucify me, just askin'. 

[apc]

On 4/25/2021 at 1:36 PM, Mazzspeed said:

Is this part of Fujinet-PC yet? Don't crucify me, just askin'. 

Yes ... for couple of last minutes

You can just grab above attached file and replace old autorun.atr file - rename autorun-zx0.atr to autorun.atr and put it into fujinet-pc/build directory (if running fujinet from there)

 

[Mazzspeed]

1 hour ago, apc said:

Yes ... for couple of last minutes

You can just grab above attached file and replace old autorun.atr file - rename autorun-zx0.atr to autorun.atr and put it into fujinet-pc/build directory (if running fujinet from there)

 

Excellent, cheers apc!

[tsom]

Just wanted to come and say, I just updated my FujuNet, and the fast config load even on my stock 800xl is FANTASTIC! Great job!

[+x=usr(1536)]

On 4/29/2021 at 6:33 AM, tsom said:

Just wanted to come and say, I just updated my FujuNet, and the fast config load even on my stock 800xl is FANTASTIC! Great job!

Ditto - somehow missed the firmware release on Wednesday and only just noticed it now.  HUGE improvement in load time, and the splash screen is a really nice touch.  Thank you!

[+DrVenkman]

Updated my units this afternoon. REALLY like the splashscreen. Well done, folks. 

[Mazzspeed]

The splash screen is quite sexeh and the added speed is appreciated. Good job!

[tschak909]

Yup, and indeed, I _really_ need to point something out, here:

 

The FujiNet core team didn't do this work. @apc did.

 

He did it, and said, "Look what I did!"

 

and we went, "WOW!"

 

and we worked with him to fold it in.

 

If you have something cool you want to add? Do it. Show everybody how cool it is, we'll work with you to fold it in!

 

-Thom

 

 

[bocianu]

New compressed loader loads much much faster. Good job, thanks!

And finally the logo I drew was used somewhere

[Mazzspeed]

14 hours ago, tschak909 said:

Yup, and indeed, I _really_ need to point something out, here:

 

The FujiNet core team didn't do this work. @apc did.

 

He did it, and said, "Look what I did!"

 

and we went, "WOW!"

 

and we worked with him to fold it in.

 

If you have something cool you want to add? Do it. Show everybody how cool it is, we'll work with you to fold it in!

 

-Thom

 

 

Open source FTW. It's a beautiful thing when collaboration results in great things...

 

Now, if only my coding skills were better!

[apc]

Thank you for nice feedback, support from AA members and to all interested into making FujiNet the real thing!

On 4/21/2021 at 12:26 AM, apc said:

Credits: @xxl for ZX0 decompression port to 6502 and boot loader @HiassofT for high speed SIO @bocianu for FujiNet banner @pirx for compression ideas

🙂 Jan

 

11 hours ago, Mazzspeed said:

Now, if only my coding skills were better!

You can start anytime! I am always surprised I can do LDX addr,Y