AspeQt: Yet another SIO2PC/APE clone

[w1k]

what about R: support? :/

[BillC]

what about R: support? :/

If you want this feature added why don't you submit a request ticket at http://sourceforge.net/p/aspeqt/tickets/

You must be logged in to do so.

[flashjazzcat]

Thank you for the insight, I tend to think that the issue is with the buffering given the fact that the file loads correctly when the SDX library is not used (no buffering). I will re-check the code I implemented to handle double sector read.

Take my theories on SDX buffering with a pinch of salt... it's largely speculation. If you want detail on the SDX side, best ask Konrad.

[drac030]

You can think of mini-buffers as being at the application side of things (as opposed to the driver side). In other words, they may only affect data which are visible by a program, not sector addresses (regardless of their nature, i.e. whether they're AtariDOS-style sector links or SpartaDOS-style file maps).

 

I presume that the "folder image" presents itself to the SDX as an AtariDOS disk. In this case, a sudden jump to the beginning of the file is hard to explain, because it can only mean that the ATARIDOS.SYS driver has been served a sector containing a circular link pointing back to the initial sector of the file.

 

But again, it is hard to explain, why the same effect does not occur when the same file is loaded in the same circumstances (I hope?), just without using X.COM. As I said above, mini-buffers can't be to blame here, because they do not see any sector links any time at all.

 

For now I can only assume that the problem, as long as it is not reproducible on a real floppy or an ATR floppy image, lies at the side of folder image emulation, and the most obvious cause may be the one fjc already suggested, i.e. SDX being served different data associated with the same sector number, which causes buffer cache inconsistency.

[flashjazzcat]

Thanks Konrad. I'm pretty interested in knowing what's different here about loading with "X" myself. I just tried using my own implementation, and the results were interesting considering what you said above. When testing my year-old implementation, I discovered that the standard loader appears to re-read the first sector of the file after 151 sector reads. In my version, this has the undesired effect of causing the entire file to be read again from the beginning. The loader - on the second "pass" - reads all 171 sectors of the 21,268 byte file and executes it. This behaviour is puzzling enough, although at least the file executes, albeit after the loader has read the complete file almost twice over. The key point is that the cache is not invalidated, since all sector numbers remain unique and are actually offsets into the file.

 

Now, using the "official" version of Aspeqt, the same request for the first sector of the file appears to happen after exactly 151 sector reads. However, since the sector number does not actually correspond to the file position here, the file pointer is set to whatever atariFiles[atariFileNo].pos contains after the first sector is read for a second time:

    /* Data sectors */
    if ((sector >= 369 && sector <= 719)) {
        int size, next;
        if  (sector >= 369 && sector <= 432)
            atariFileNo = sector - 369;

        if (!atariFiles[atariFileNo].exists) {
            data = QByteArray(128, 0);
            return true;
        }
        if (sector == atariFiles[atariFileNo].lastSector)
            atariFiles[atariFileNo].pos = atariFiles[atariFileNo].prevPos;

Using the example file, after 151 sector reads, sector 407 is requested again (which represents the first sector of the 39th file in the mirrored folder). AtariFileNo is set to 38, but I wonder if atariFiles[atariFileNo].pos is not reset to zero, perhaps resulting in the wrong data being delivered for that sector and all ensuing sectors.

 

If atariFiles[atariFileNo].pos was reset to zero when the first sector was re-requested, it would also throw the file position out of whack if subsequent reads were not contiguous (although it would work if they were), so it's difficult to see an elegant way of fixing the issue when the sector number does not directly correspond to the file position.

 

Meanwhile, although my code works after a fashion, I need to figure out why there are so many redundant sector reads when the standard loader re-reads the first sector of the executable. Using "X", meanwhile, results in no such repetition, since the first sector of the file does not appear to be re-read using X.COM.

 

Can you confirm why the standard SDX loader would appear to re-read the first sector of the executable after c. 19,000 bytes, Konrad? Or are we being thrown a red-herring here?

 

Note: Ray's "double-read" fix comes into play when the file is copied, but that special-case fix only works when the same sector is requested twice in a row. Perhaps if the same sector is requested non-consecutively, incorrect data is returned. Note also that the double-read fix can be dispensed with if the sector numbers are mapped directly to the file position.

 

EDIT: Curiouser and curiouser. It appears the behaviour of my version may not be a bug after all. I just tried loading the sample executable without "X" in Altirra from a mirrored folder with the sector counter on, and in Avery's implementation, once again after 151 sectors, SDX reads the whole executable right from the beginning again. I wondered, in fact, why standard A8 binaries seemed to be taking a long time to load when launched without "X"...

Edited December 19, 2013 by flashjazzcat

[drac030]

The loader cannot "re-read the first sector after 151 sector reads", because the loader has no clue about sectors: it only sees a file and reads it sequentially (no seeks etc.). But the behaviour you describe is weird indeed. Could you confirm it for an ATR image?

[atari8warez]

I am just reading your comments so far, the issue is indeed getting stranger as the discussion progresses. I too at this point have a feeling that the duplicate sector request may be the culprit if the duplication does not occur consecutively but randomly during the whole read process. I do have an idea to cover such duplications that I can implement and play with and see what happens. I am just busy right now assembling SIO2PCs, however I will look into this as soon as I am finished with the hardware commitments. I am still baffled though with the fact that this problem does not occur with the X loader.

Edited December 19, 2013 by atari8warez

[phaeron]

This is due to a combination of suboptimal behaviors in both ATARIDOS.SYS and the standard SpartaDOS X loader.

 

Let's use a simple test executable consisting of 4K init segments:

 

    .rept 10
    org    $8000
:$1000 dta $00
    rts
    ini    *-1
    .endr

    org    $8000
main:
    rts

    run main

 

We can put this executable into a folder and mount that folder in Altirra as a virtual DOS 2 disk. When loaded from there, the VDISK logging channel indeed shows SpartaDOS X restarting the read of the file:

 

 

VDISK: Reading sector 302 (sector 228 of file 35 / big.obx)
VDISK: Resurrecting cached link at sector 302
VDISK: Reading sector 303 (sector 229 of file 35 / big.obx)
VDISK: Resurrecting cached link at sector 303
VDISK: Reading sector 304 (sector 230 of file 35 / big.obx)
VDISK: Resurrecting cached link at sector 304
VDISK: Reading sector 39 (sector 0 of file 35 / big.obx)
VDISK: Resurrecting cached link at sector 74
VDISK: Reading sector 75 (sector 1 of file 35 / big.obx)
VDISK: Resurrecting cached link at sector 75
VDISK: Reading sector 76 (sector 2 of file 35 / big.obx)

 

To explain this behavior, we can compare it against the same file being loaded through the PCL: device. The PCL: device simply marshals the SpartaDOS X I/O commands over the SIO bus, so by using Altirra's emulated PCLink device and PCLINK logging channel we can see what I/O calls the loader is issuing:

 

 

PCLINK: Read at pos 24638/41083, len 16, actual 16
PCLINK: Read at pos 24642/41083, len 2, actual 2
PCLINK: Read at pos 24644/41083, len 16, actual 16
PCLINK: Read at pos 24648/41083, len 4097, actual 4097
PCLINK: Read at pos 28745/41083, len 16, actual 16
PCLINK: Read at pos 28749/41083, len 2, actual 2
PCLINK: Read at pos 28751/41083, len 16, actual 16
PCLINK: Read at pos 28755/41083, len 4097, actual 4097

 

Notice that the loader is issuing 16 byte reads to fetch the headers even though it only needs 4 bytes. This means that it has to seek backwards in order to read the segment. In this section, there is a read of 16 bytes from position 28745 which spans two sectors, sectors 229 and 230 of the file. The subsequent read is then at position 28749, which starts at sector 229. This means that the loader is trying to seek backwards one sector in a file on a DOS 2 disk, where the sectors are only forwardly linked. ATARIDOS.SYS is apparently unable to cache the last sector and instead has to follow the entire block chain from the beginning again. On an SDFS disk, this is much less costly as only the sector map chain must be walked and it is doubly linked. This is presumably why the manual warns about executable loads being much slower on an Atari DOS disk.

 

If we repeat the load process with X.COM, we can see that it is much simpler:

 

 

PCLINK: Read at pos 24648/41083, len 4097, actual 4097
PCLINK: Read at pos 28745/41083, len 2, actual 2
PCLINK: Read at pos 28747/41083, len 2, actual 2
PCLINK: Read at pos 28749/41083, len 2, actual 2
PCLINK: Read at pos 28751/41083, len 2, actual 2
PCLINK: Read at pos 28753/41083, len 2, actual 2
PCLINK: Read at pos 28755/41083, len 4097, actual 4097

 

X.COM only reads headers 2 bytes at a time and only reads sequentially without gaps or overlaps, so there is no problem.

[flashjazzcat]

Thanks Avery - I had a feeling you'd be able to explain it.

 

I do have an idea to cover such duplications that I can implement and play with and see what happens.

Mapping the sector numbers to the file position deftly "covers all the bases" without making the whole thing needlessly complex. For larger file support perhaps an SDFS implementation would be a good idea.

Edited December 20, 2013 by flashjazzcat

[drac030]

 

Notice that the loader is issuing 16 byte reads to fetch the headers even though it only needs 4 bytes. This means that it has to seek backwards in order to read the segment.

 

 

Right, only except that's not directly the loader what is issuing the 16-byte reads: that's the mini-buffer code (so it seems it is to blame here after all). I somehow forgot about it, but a non-minibuffered call (FREAD) following a mini-buffered call (FGET) indeed cancels the mini-buffer contents and seeks backwards. On SDFS such a seek indeed costs little.

 

[flashjazzcat]

Ah well - so there was something to be said for my murky speculation and half-remembered facts after all.

[flashjazzcat]

The ATARIDOS.SYS backward seek overhead is even worse with a different (smaller) binary: a 14,664 byte program incurred a grand total of 568 sector reads, which is 72,704 bytes transferred (including sector links; 71,000 bytes of actual data) just to execute the file.

[drac030]

That looks pretty bad, but fortunately should be easy to lift (in next SDX release).

 

I have been using Atari DOS diskettes only to copy stuff, so I did not even realize that there can be such a problem.

[+Stephen]

Thanks guys - your (all of your) dedication to fixing bugs is great to see!

[drac030]

That is not a real bug, that is some simplification of the way the minibuffers work. Of course, the code should get the data remaining in a minibuffer instead of cancelling its contents and reading the relevant portion again from media, but I can guess that the original author either was going to improve this later or decided to tradeoff some (on SDFS the effect is hardly visible) loading speed for the code's size.

Edited December 22, 2013 by drac030

[flashjazzcat]

I liked Avery's tactful term for it: sub-optimal. Good that it surfaced, one way or another, although it's completely understandable that it went unnoticed all these years.

[w1k]

if i mount directory, can i copy files from atari to them?

[flashjazzcat]

Nope: it's read-only.

[atari8warez]

Ok folks here is a beta preview of the upcoming AspeQt v.1.0. With this version's final release, I am planning to take AspeQt from the BETA status to a hopefully stable and mature final release. That doesn't mean it will have every imaginable functionality of an APE, but it will no longer be in perpetual BETA status

 

The work on v1.0 will take a while and I am planning to make several executable only test releases for the existing users to test it out and report any bugs or make suggestions for improvements. I am currently evaluating and implementing change requests and bug fixes according to existing tickets in the software repository.

 

Any new requests and bug reports must ultimately be recorded using the Sourceforge ticket system if they have to be considered seriously as I have no time or energy to wade through multitudes of Atariage message threads (including this one) to figure them out.

 

The test executable will only be available for Windows as this is the platform I develop for, I apologize to linux users/potential testers but I currently have no means to compile code on a linux box. The final release will also see source code uploaded to its usual repository.

 

Here's what's new in this preview release:

version 1.0.0 (Preview-1) (Jan 26, 2014)
    * Changed/fixed some UI elements
    + Added an option to Start/Stop Printer Emulation
        Turning the emulation OFF will allow the user to use an Atari connected printer instead of a PC printer.
        This switch affects the currently running session of AspeQt only and is not permanent between sessions
    + Added an option to hide/show drives 9-15
        This switch is either permanent or temporary depending on user preference, and can be:
        - temporary for the currently running session or,
        - globally permanent (for all sessions of AspeQt running on the same computer) or,
        - permanent for select sessions of AspeQt (when AspeQt is run from a session file)
    - Removed the half-baked ATX file support code in favour of using VAPI.DLL, as the author who started 
        the work on ATX support is no longer interested in developing it further. 
        VAPI.DLL support will be added before v1.0 final release

aspeqt_v1_P1.zip

Edited January 27, 2014 by atari8warez

[marcokitt2000]

 

Ok folks here is a beta preview of the upcoming AspeQt v.1.0. With this version's final release, I am planning to take AspeQt from the BETA status to a hopefully stable and mature final release. That doesn't mean it will have every imaginable functionality of an APE, but it will no longer be in perpetual BETA status

 

The work on v1.0 will take a while and I am planning to make several executable only test releases for the existing users to test it out and report any bugs or make suggestions for improvements. I am currently evaluating and implementing change requests and bug fixes according to existing tickets in the software repository.

 

Any new requests and bug reports must ultimately be recorded using the Sourceforge ticket system if they have to be considered seriously as I have no time or energy to wade through multitudes of Atariage message threads (including this one) to figure them out.

 

The test executable will only be available for Windows as this is the platform I develop for, I apologize to linux users/potential testers but I currently have no means to compile code on a linux box. The final release will also see source code uploaded to its usual repository.

 

Here's what's new in this preview release:

version 1.0.0 (Preview-1) (Jan 26, 2014)
    * Changed/fixed some UI elements
    + Added an option to Start/Stop Printer Emulation
        Turning the emulation OFF will allow the user to use an Atari connected printer instead of a PC printer.
        This switch affects the currently running session of AspeQt only and is not permanent between sessions
    + Added an option to hide/show drives 9-15
        This switch is either permanent or temporary depending on user preference, and can be:
        - temporary for the currently running session or,
        - globally permanent (for all sessions of AspeQt running on the same computer) or,
        - permanent for select sessions of AspeQt (when AspeQt is run from a session file)
    - Removed the half-baked ATX file support code in favour of using VAPI.DLL, as the author who started 
        the work on ATX support is no longer interested in developing it further. 
        VAPI.DLL support will be added before v1.0 final release

Hello Atari8warez,

 

Nice to hear about this i like this program maybe in the further it become beter than ape & prosystem for free.

 

Gr. Marco

Edited January 27, 2014 by marcokitt2000

[atari8warez]

Hello Atari8warez,

 

Nice to hear about this i like this program maybe in the further it become beter than ape & prosystem for free.

 

Gr. Marco

 

Hi Marco, my approach to this whole venture is to follow the footsteps of the original developer of AspeQt, in that respect, I try to add user friendly and easy to use features rather than make AspeQt look like a techie tool. When I decide on what features to add, I try to put myself in the majority of Atari user's shoes. Most of us don't really care about multitude of fine adjustments some other similar software may offer and I really have a big doubt about their usefulness, so don't expect to see those kinds of features in AspeQt, as for as functionality goes, I also take the same approach there. Other software may have additional features that maybe only a handful of users find useful (an example might be Epson printer emulation - how many people today print documents or graphics from their Atari printer, really?) so in my opinion a simple text printer emulation which directs the Atari output to a PC printer is all most of us need.

 

Having said that, I am always open to suggestions and requests and if I see enough interest for a feature I'll do my best to implement it. So it is important for AspeQt users to show their interest by logging their suggestions, bug reports and feature requests not only here in this forum but also officially through the ticket system in Sourceforge

 

I personally don't think that "best things in life are free"... but we can come close in some situations, I hope that AspeQt may one day become one of these exceptions

[+JAC!]

Minor feedabck:

- Some menu entry captions are not consistely sentence case ("Stop Printer" / "Show printer") etc. in English

- "Clear messages" tooltip is not adapted when language is switched

Edited January 28, 2014 by JAC!

[atari8warez]

Minor feedabck:

- Some menu entry captions are not consistely sentence case ("Stop Printer" / "Show printer") etc. in English

- "Clear messages" tooltip is not adapted when language is switched

Thanks for the feedback JAC, will fix the first one, language translations on the other hand depend on other people volunteering for the translation work , I only do Turkish translation myself as it is my native language, plus translations are usually done just before the final release of the particular version.

Edited January 28, 2014 by atari8warez

[atarixle]

cool to hear that good news!

[+JAC!]

>- "Clear messages" tooltip is not adapted when language is switched

This does not refer to the actual translation, but to the text not being adapted when the language is switched in the preferences. The correct text shows up after a restart of the program. For the other tooltips it's OK.