Jump to content
apc

FujiNet Config loading faster

Recommended Posts

Posted (edited)

Hi,

 

I put some asm code together to make Config loader:

 

912295576_Screenshotfrom2021-03-1602_45_12.thumb.png.13c05aac045da7fe9b65e5fc9eaa7163.png

 

It uses HSIO routines by HiassofT for faster load (on not modded Atari).

Short video: https://youtu.be/kgvbemvtdDg

 

In case of interest I will prepare disk image to test and share the code.

 

Jan

Edited by apc
  • Like 10

Share this post


Link to post
Share on other sites
6 minutes ago, apc said:

Hi,

 

I put some asm code together to make Config loader:

 

912295576_Screenshotfrom2021-03-1602_45_12.thumb.png.13c05aac045da7fe9b65e5fc9eaa7163.png

 

It uses HSIO routines by HiassofT for faster load (on not modded Atari).

Short video: https://youtu.be/kgvbemvtdDg

 

In case of interest I will prepare disk image to test and share the code.

 

Jan

Awesome! I use Hias's routines that are a part of the U1MB, but this would make config load faster for everyone! Excellent. :)

 

-Thom

 

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites

To test it, attached are versions with high speed SIO autorun-cloader.atr included and without it autorun-cloader-nohs.atr. Put it on your TNFS server or SD card and insert image into disk slot 1 and boot.

 

I think for the version with HSIO included it would be nice to detect if the SIO is already patched, and then skip bundled routines if patched SIO is detected. How to easily detect patched SIO, check the content on $E459, other location(s)?

 

Source code:

https://github.com/a8jan/fujinet-config-loader

 

Jan

  • Like 1

Share this post


Link to post
Share on other sites
12 hours ago, apc said:

To test it, attached are versions with high speed SIO autorun-cloader.atr included and without it autorun-cloader-nohs.atr. Put it on your TNFS server or SD card and insert image into disk slot 1 and boot.

 

I think for the version with HSIO included it would be nice to detect if the SIO is already patched, and then skip bundled routines if patched SIO is detected. How to easily detect patched SIO, check the content on $E459, other location(s)?

 

Source code:

https://github.com/a8jan/fujinet-config-loader

 

Jan

Paging @HiassofT ^^

Share this post


Link to post
Share on other sites

The easiest way is to check the contents of $CFF0-$CFFF. If the OS is patched the first 4 bytes contain the string "Hias", followed by version number and date. eg

48 69 61 73 20 31 2e 33 32 20 32 30 30 34 32 38  Hias 1.32 200428

so long,

 

Hias

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites
2 hours ago, HiassofT said:

The easiest way is to check the contents of $CFF0-$CFFF. If the OS is patched the first 4 bytes contain the string "Hias", followed by version number and date. eg

48 69 61 73 20 31 2e 33 32 20 32 30 30 34 32 38  Hias 1.32 200428

so long,

 

Hias

So wait, the high speed code can't work on the 400/800 ? 

-Thom

Share this post


Link to post
Share on other sites
Posted (edited)

The highspeed code works fine on 400/800 but the ROM patch needs about 1k of ROM space - the only possibility to get that much space was to reuse the international character set.

 

Actually it's possible to patch the 400/800 OS, patchrom for the PC supports that, but it will create a 16k ROM - you can use that in an XL or probably also in an incognito, but not on a stock 400/800.

 

so long,

 

Hias

Edited by HiassofT

Share this post


Link to post
Share on other sites

@apc this is fantastic!

the process could be sped up even more by having CONFIG.COM compressed, with e.g. exomizer

Share this post


Link to post
Share on other sites
Posted (edited)
37 minutes ago, pirx said:

@apc this is fantastic!

the process could be sped up even more by having CONFIG.COM compressed, with e.g. exomizer

As long as whatever method is used will also work on 400/800 systems... That is a hard requirement for me, I see no reason to exclude 1/3rd of the installed base just because somebody thinks that the only systems that exist are 130XE's... ;)

 

-Thom

 

Edited by tschak909
  • Like 3

Share this post


Link to post
Share on other sites
Posted (edited)

exomizer cuts the baby better than in half. i do not know if it works, but proof of concept is worth exploring!

command: 

wine exomizer-3.1.0/win32/exomizer.exe sfx sys -t168 CONFIG.COM

CONCOMPR.COM

Edited by pirx
  • Like 1

Share this post


Link to post
Share on other sites

Hello guys

 

There are more high speed routines than the one Hias wrote.  Some OSses even have high speed code built in.  There must be some way to check for high speed code without checking for specific high speed code.

 

Sincerely

 

Mathy

 

  • Like 2

Share this post


Link to post
Share on other sites
On 3/20/2021 at 3:12 AM, tschak909 said:

As long as whatever method is used will also work on 400/800 systems...

Config Loader should work on 400/800 as far as there is enough RAM for Config itself ;-) I cannot test myself, I am not owner of those diamonds. At least I tested with Altirra. The Config was successfully fast loaded by Config Loader on 400 with 48K RAM.

 

On 3/20/2021 at 3:43 PM, pirx said:

exomizer cuts the baby better than in half. i do not know if it works, but proof of concept is worth exploring!

I like the idea but the attached crunched COM file does not work for me. I am trying to load it as regular COM/XEX. I'm doing something wrong?

 

14 hours ago, Mathy said:

There are more high speed routines than the one Hias wrote.  Some OSses even have high speed code built in.  There must be some way to check for high speed code without checking for specific high speed code

Yes, check one by one all possible specific speed code routines 🙂

Really not sure if it's doable (in simple way) to detect all possible routines. It may not be necessary to check at all ...

The Config Loader bundled with HiassofT's routines is calling those routines only when loading Config, bypassing ROM routines (stock or modded) just to get this one program loaded. No patch is installed, no vector is modified. When Config is started, ROM routines (stock or modded) are used again. Is it a problem?

 

Jan

 

Share this post


Link to post
Share on other sites
1 hour ago, apc said:

I like the idea but the attached crunched COM file does not work for me. I am trying to load it as regular COM/XEX. I'm doing something wrong?

yes, it does not work, i haven't got energy to check it, so this is only a silly proof of concept (that compression is very effective for the current CONFIG.COM. I've never used exomizer sfx, but was compressing/decompressing the data myself.  I'll look a bit where it is loading, possibly some conflict arises. And yea, CONFIG.COM does not work on A400 16KiB, so 48KiB is the right target.

Share this post


Link to post
Share on other sites
Posted (edited)

CONF_EX.COM

Yea, this one works, I used fatastic (albeit a bit tricky to use) Superpacker from the A8 god TeBe: http://madteam.atari8.info/uzytki/sp.7z

Superpacker works like a charm under wine.

 

But when looking on the font change I am afraid it uses XL RAM, so no good for production use due to Thomas' (reasonable) requirements.

Edited by pirx

Share this post


Link to post
Share on other sites

Ha! XL RAM usage is configurable, moved the data around and this should do the job. Turned off decrunch effect. Turning off the screen would get it a bit faster, too.

CONF_EX3.COM

Share this post


Link to post
Share on other sites

There is a version with loader and compressed Config:

autorun-cloader-zapped.atr

 

However I'm no sure I used SP properly. Compressed config:

config.xex

 

The result is.. we saved 3 seconds to load smaller file and it takes 2 seconds to get it decompressed ;-) (divisor 0)

 

Some measured numbers to load the Config:

 

  original Config at 19200 = 15s

  compressed Config at 19200 = 8s

  Loader + compressed Config at 19200 = 9s (cost of banner)

 

  Loader + Config, divisor 0 = 6s

  Loader + compressed Config, divisor 0 = 5s

 

  Loader + Config, divisor 8 = 10s
  Loader + compressed Config, divisor 8 = 7s

 

Both methods nicely boosts the Config loading, best when combined 🙂

 

Jan

  • Like 2

Share this post


Link to post
Share on other sites
Posted (edited)

nice :) with divisor=0 depacking is barely faster than loading. 

exomizer is not the fastest in depacking, possibly lz4 would be better. It compressess the most, but it might be more optimal to go for a bit worse compression and a faster decompression. Also turning off the screen for depacking could cut decompression time ~20-30%

Edited by pirx

Share this post


Link to post
Share on other sites

Hello apc

 

6 hours ago, apc said:

Yes, check one by one all possible specific speed code routines 🙂

Really not sure if it's doable (in simple way) to detect all possible routines.

 

That's not what I meant.  There has to be a way to test of the computer is going to use high speed.  If you test for the presence of a routine, you are likely to forget one (or somebody comes up with a new one) and you probably need space for going through all the possible routines.

 

Sincerely

 

Mathy

 

Share this post


Link to post
Share on other sites

Hi Mathy,

 

Understand. Some code for detection can be prepared, tested, etc. The point is, I hope, I don't really need to detect it for the purpose of the Config Loader (as explained above). I thought it would be nice to detect and skip routines bundled with Loader. For now I will keep it as nice to have if no arguments to change it to must have ;-)

 

Jan

Share this post


Link to post
Share on other sites
Posted (edited)

years ago we did something like this in our version of MicroSpartaDos (checking for various turbo routines), but later we found out basically the only one making any sense was the one with '?' command (Speedy?)

 

oh, also it would be better to load compressed data to higher memory by setting `load_addr` to $6c00, one memory copy less.
and `buffer` to $9100, so it starts with BASIC ON.

added LZ4 compressed file, maybe it will get faster?

 

-----------------------------------------

and now for something (barely) completely different:

it should be possible to glue the turbo routine and banner into a single file with compressed CONFIG.COM for an untimate pleasure and speed-up - one file looku-up less :))))

if @tschak909 gets a command line commands for his makefile he could incorporate this development easily into the official FJN firmware :]

 

 

conf_ex4.xex

conf_lz4.xex

Edited by pirx
  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)
23 hours ago, pirx said:

Attached LZ4 is winner in decompression speed, but the file is slightly larger. EX4 is smaller but slower to decompress (seems it does not work with BASIC ON). Now it's hunt for deci-seconds 🙂

 

ATR with the above LZ4 packed Config:

autorun-cloader-lz4.atr

 

23 hours ago, pirx said:

it should be possible to glue the turbo routine and banner into a single file with compressed CONFIG.COM for an untimate pleasure and speed-up - one file looku-up less :))))

Agree, further optimization possible ;-) - banner is currently loaded as uncompressed bitmap, sector 361 is read twice (by PicoBoot, then by Loader), etc. On the other hand, I am not sure I want to spend much time with fine tuning... E.g. If bundled together, the result might not be able to run as standalone program from DOS anymore, PicoBoot routines called by Loader will not be present in memory. Solution would be to include those routines into Loader, increase it's size, compress it, etc. etc .. Looks like extra effort with some chance to save 1 sec. Will keep it on TODO list and glad to support if anyone can explore and progress in this area of improvements :)

 

23 hours ago, pirx said:

if @tschak909 gets a command line commands for his makefile he could incorporate this development easily into the official FJN firmware :]

@pirx Can you provide packer program command line parameters? @tschak909 for the config/autorun.atr build process, if some utility should be added (like packer), do you need Linux, WSL or native Windows binaries?

 

Jan

Edited by apc

Share this post


Link to post
Share on other sites
14 minutes ago, apc said:

Attached LZ4 is winner in decompression speed, but the file is slightly larger. EX4 is smaller but slower to decompress (seems it does not work with BASIC ON). Now it's hunt for deci-seconds 🙂

 

ATR with the above LZ4 packed Config:

autorun-cloader-lz4.atr 90.02 kB · 0 downloads

 

Agree, further optimization possible ;-) - banner is currently loaded as uncompressed bitmap, sector 361 is read twice (by PicoBoot, then by Loader), etc. On the other hand, I am not sure I want to spend much time with fine tuning... E.g. If bundled together, the result might not be able to run as standalone program from DOS anymore, PicoBoot routines called by Loader will not be present in memory. Solution would be to include those routines into Loader, increase it's size, compress it, etc. etc .. Looks like extra effort with some chance to save 1 sec. Will keep it on TODO list and glad to support if anyone can explore and progress in this area of improvements :)

 

@pirx Can you provide packer program command line parameters? @tschak909 for the config/autorun.atr build process, if some utility should be added (like packer), do you need Linux, WSL or native Windows binaries?

 

Jan

source code. I do dev on WSL, but @mozzwald may use something different.

-Thom

 

Share this post


Link to post
Share on other sites
4 hours ago, apc said:

if some utility should be added (like packer), do you need Linux, WSL or native Windows binaries?

The release firmware from fujinet.online is built on Linux.

Share this post


Link to post
Share on other sites

Updated disk image with handcrafted Config Loader using LZ4 boot code aka "Frankenstein loader" 🙂

 

autorun-lz4.atr

 

Config load time: 3.5s with divisor 0 or 4.5s with divisor 8

 

Big thanks to @xxl 🙂 for his LZ4 boot code!

 

Jan

 

NB: Need to find a way to automate the build process of this baby and think how to make it runnable (not only boot-loaded) under Frankenstein not-ready DOSes ;-)

  • Like 5

Share this post


Link to post
Share on other sites

🙂 nice.

 

15 minutes ago, apc said:

NB: Need to find a way to automate the build process of this baby and think how to make it runnable (not only boot-loaded) under Frankenstein not-ready DOSes ;-)

LiteDOS 3.x reads LZ4 binary/executables so... it could be done 🙂

  • Like 3

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...