Jump to content
IGNORED

RespeQt: Fork of AspeQt 1.0.0-preview7/r79


Joey Z

Recommended Posts

 

 

Your prior suggestion of inserting small delays in the SIO sequence to allow time for transitions appears to have helped a great deal. I managed to reduce loading times for my go-to test .atr from about 6 - 10 pauses (seven seconds each, by the way!) to about 2 - 3. But it's all empirical hit-or-miss - insert a short delay here, compile, test, repeat ... There's absolutely nothing scientific about it at all. It just seems like a nasty hack, you know? There ought to be a better way to go about this, especially as the two other OS X-compatible peripheral emulators aren't affected (SIO2OSX and SIOServer).

 

So thanks for that pointer. I'll look into that over the next few days and see what I can dig up.

 

Oh, and for Joey Z, that reference to AspeQt in sioworker.cpp is at line 217. I've already changed it in my working copy. :)

 

Actually, I specifically left that reference on purpose. It's a reference to the AspeQt Client which is still *exactly* the same as it was before, and I don't have source for it, so it remains the AspeQt Client. That's on my TODO list of stuff to do, figure out what to do about the AspeQt client...

 

Obviously looking far in to the future - I am not agitating for this feature, just asking if it is possible - would 'RespeQT' be able to load custom ROMs, such as the 'USDoubler', 'Happy', 'Speedy' upgrades and so forth? You would be effectively building a software version of the 'MegaSpeedy' board. Were that possible then an Android build supporting custom firmware would allow one to completely dispense with a real '1050' by simply using a £20 tablet from Amazon and a £12 DIY SIO2PC! That would be the best of all worlds!!!

 

In the past Phaeron has mentioned how time-consuming and tricky it would be to add this type of functionality to "Altirra". I wonder if it might be easier to implement with a dedicated floppy emulator like "RespeQT"?

 

morelenmir, the issue with that is you'd have to do low level emulation of the 1050. That is, emulate the 6502 inside, and all the hardware. AspeQt is not a 1050 emulator, it's an SIO device emulator, at a pretty high level. This would require a lot of code to implement, including a full 6502 emulation core, and also a WD1772 emulation core I believe. Also, I'm not really sure of the practical advantages of that. It seems to me it'd just be a 'cool thing' that it does. What does any of these upgrades do that you can't do through an added emulation feature? nothing. What does any of these do that RespeQt doesn't currently do? plenty, but I don't have any idea, which brings up a good point:

 

If you have feature requests or suggestions, let me know.

Edited by Joey Z
  • Like 1
Link to comment
Share on other sites

Actually, I specifically left that reference on purpose. It's a reference to the AspeQt Client which is still *exactly* the same as it was before, and I don't have source for it, so it remains the AspeQt Client. That's on my TODO list of stuff to do, figure out what to do about the AspeQt client...

 

Good to know. Thanks!

 

What does any of these do that RespeQt doesn't currently do? plenty, but I don't have any idea, which brings up a good point:

 

If you have feature requests or suggestions, let me know.

How 'bout R: device emulation or something along those lines, allowing access to Telnet BBS systems through the host's Internet connection? Another cool thing to do would be to allow creating an .atr image from a real floppy if you have an interface device that can do it, and writing .atr images back to physical media, but that might take too much work.

 

One thing that would absolutely be helpful would be an option to toggle more detailed logs. For instance, as I fart around trying to optimize the OS X port, it would be very helpful for RespeQt to tell me WHY it has to "Read Sector xyz (128 bytes) ..." twice - what caused the first misread, and why is the code spinning its wheels for seven seconds or so every time that happens? Etc.

Edited by DrVenkman
  • Like 1
Link to comment
Share on other sites

How 'bout R: device emulation or something along those lines, allowing access to Telnet BBS systems through the host's Internet connection? Another cool thing to do would be to allow creating an .atr image from a real floppy if you have an interface device that can do it, and writing .atr images back to physical media, but that might take too much work.

 

One thing that would absolutely be helpful would be an option to toggle more detailed logs. For instance, as I fart around trying to optimize the OS X port, it would be very helpful for RespeQt to tell me WHY it has to "Read Sector xyz (128 bytes) ..." twice - what caused the first misread, and why is the code spinning its wheels for seven seconds or so every time that happens? Etc.

.atr image from a real floppy with 10502PC would be best done in a separate program. There's not a whole lot shared by a disk emulator and a disk imager program. R: emulation is already planned, other things just have more priority right now, and I'm limited on time now that I've started working at an internship.

  • Like 1
Link to comment
Share on other sites

In case anyone (Mac users) are interested in my work-in-progress, here is an update: I've had to modify Joey Z's RespeQt r1 code to add conditional compile blocks around the delay code snippets I've introduced in a bunch of places in an effort to both reduce CPU usage from it's crazy-high levels (thanks HiassofT) and smooth out some of the stalling during .atr I/O operations (thanks phaeron). It's ugly and makes me feel unclean but it helps. It's still just a hack though - there HAS to be a better way but it's gonna take me a long time to grasp C++ I/O operations and how they're handled in OS X in order to fix the the underlying problem(s). I really wish Mellis (author of the now-obsolete SIO Server app for OS X) would jump in here and fix this spaghetti code.

 

Anyway, loading executables (.xex) files appears to work flawlessly, and 256 byte sectors seem to suffer few stalls than do 128 byte sectors. However, since most .atr files out in the wild are single-density formats, this is kind of big. Worse, the stalling is non-deterministic - the same file might load with only 1 or 2 stalls one time, but reboot the machine and you'll get hit for 4 or 5 delays, on different sectors. My go-to test file is an .atr I made myself of my original 1980's Zork I disk from back in the day that I imaged myself last summer. The disk loads fine in emulators and using an old Win version of AspeQt, so I know the file is fine.

Link to comment
Share on other sites

Here is a windows build of r0 and a prelease of r1. I'd appreciate it if someone can give ApeTime a try.

 

Joey, I guess I didn't mention it, but ApeTime in r1 works fine in the OS X builds I've been making. I've got an .atr boot disk image of SpartaDOS 3.2g that has the client program on it; when you boot the disk, it installs a status line at the top of the screen with the date/time pulled in from the RespeQt server.

Link to comment
Share on other sites

 

Joey, I guess I didn't mention it, but ApeTime in r1 works fine in the OS X builds I've been making. I've got an .atr boot disk image of SpartaDOS 3.2g that has the client program on it; when you boot the disk, it installs a status line at the top of the screen with the date/time pulled in from the RespeQt server.

do you happen to know if it's using ApeTime or the AspeCL protocol though? I think it would say in the RespeQt logs.

Link to comment
Share on other sites

do you happen to know if it's using ApeTime or the AspeCL protocol though? I think it would say in the RespeQt logs.

 

I just tested and here's what RespeQt told me as the screen updated the date/time on my 800XL:

 

"[APE time downloader] Read date/time (7/11/15 6:52 PM)."

Link to comment
Share on other sites

Holy shit, I may have fixed the OS X problems ... Gotta do more regression testing, and test a whole lot more .atr files, but my "test case" Zork I .atr, the one I imaged myself from my original 32 year old floppy I bought in 1983, appears to load and run flawlessly, as do another few disks. I also need to test binary loads and a few other things later tonight or tomorrow but I really think I may have it fixed.

 

Any interested Mac users, stay tuned ...

  • Like 2
Link to comment
Share on other sites

Holy shit, I may have fixed the OS X problems ... Gotta do more regression testing, and test a whole lot more .atr files, but my "test case" Zork I .atr, the one I imaged myself from my original 32 year old floppy I bought in 1983, appears to load and run flawlessly, as do another few disks. I also need to test binary loads and a few other things later tonight or tomorrow but I really think I may have it fixed.

 

Any interested Mac users, stay tuned ...

well, I'm getting a hackintosh setup in a VM, so maybe I can help with this. What exactly changed that seemed to fix it finally? Just messing with timing more, or something actually different in the structure of the code?

Link to comment
Share on other sites

well, I'm getting a hackintosh setup in a VM, so maybe I can help with this. What exactly changed that seemed to fix it finally? Just messing with timing more, or something actually different in the structure of the code?

 

 

The first thing I did was make the change HiassofT suggested to reduce the crazy-high 100%+ CPU usage by the program when it's listening to the serial port interface. In the QByteArray StandardSerialPortBackend::readCommandFrame() section of serialport-unix.cpp, there are two sections, one waiting for the command line to go off, another waiting for it to come back on. In two places there, the code there as carried over from AspeQt is QThread::yieldCurrentThread(). That causes the thread to take off and spike the CPU, which is decidedly not good. I changed both references to: QThread::usleep(1000). That change alone fixes the CPU spiking and immediately improved the success rate of loading .atr files. Unfortunately, that section of code is not *nix-agnostic. In other words, I don't know how changing it affects a standard Linux build. It can be surrounded by #ifdef Q_OS_MAC ... #endif blocks if necessary, however, and the original left intact.

 

Anyway, despite fixing the CPU spikes it did not completely fix the .atr file loading stalls and problems like that. I spent days adding delays of various lengths in several likely places in different modules of both the SerialPortBackend read/wite operations and in the AtariSIOBackend modules. Depending on exactly where I added the delays, and how large I made them (from 30 microseconds to several hundred), things would improve but never be completely smooth.

And then I noticed this little section, where a previous author had already inserted a medium-sized delay:

bool StandardSerialPortBackend::writeComplete()
{  
     SioWorker::usleep(300);
     return writeRawFrame(QByteArray(1, 67));
}
bool StandardSerialPortBackend::writeError()
{
     SioWorker::usleep(300); 
     return writeRawFrame(QByteArray(1, 69));
}

Hmmm. Changing those two instances to SioWorker::usleep(500); instead fixes the .atr stalling/stuttering issue completely, all by itself, with no need whatsoever for any additional delays or changes to the I/O code at all.

 

Now once again, I have no idea how this would impact Linux users; it's basically adding 200 microseconds to the tail-end of these operations, which is just enough to smooth out the reads under OS X, though apparently not needed for *nix in generally. I'm happy to add these changes into separate little conditional blocks if they affect Linux users, but it would be interesting to see if these changes that I've made negatively affect Linux users or make no difference at all.

 

Lastly, I got annoyed by a set of continuing cautions I was getting with regard to line 80 in serialport-unix.cpp. The warnings didn't affect compilation but after thinking about it a lot, the way the code is written doesn't make any sense. Here is what's there by default from AspeQt:

if (!ioctl(mHandle, TIOCMGET, &status) < 0) {

This is Boolean logic-fail at work, it seems to me. How can the result of a Boolean operator be negative? So I added parentheses around the left side of the expression to silent one caution, and changed the comparison value to "1" so the Boolean comparison actually makes sense logically, which silences the second warning I was getting on compilation:

if ((!ioctl(mHandle, TIOCMGET, &status)) < 1) {

Now having said all of this, everything I've been doing has been to fix read operations, not writes. I have done very, very little write testing of this software at all. I suppose I need to open up some productivity apps and beat on it to see if there are problems in that regard.

Edited by DrVenkman
  • Like 2
Link to comment
Share on other sites

 

 

The first thing I did was make the change HiassofT suggested to reduce the crazy-high 100%+ CPU usage by the program when it's listening to the serial port interface. In the QByteArray StandardSerialPortBackend::readCommandFrame() section of serialport-unix.cpp, there are two sections, one waiting for the command line to go off, another waiting for it to come back on. In two places there, the code there as carried over from AspeQt is QThread::yieldCurrentThread(). That causes the thread to take off and spike the CPU, which is decidedly not good. I changed both references to: QThread::usleep(1000). That change alone fixes the CPU spiking and immediately improved the success rate of loading .atr files. Unfortunately, that section of code is not *nix-agnostic. In other words, I don't know how changing it affects a standard Linux build. It can be surrounded by #ifdef Q_OS_MAC ... #endif blocks if necessary, however, and the original left intact.

 

Anyway, despite fixing the CPU spikes it did not completely fix the .atr file loading stalls and problems like that. I spent days adding delays of various lengths in several likely places in different modules of both the SerialPortBackend read/wite operations and in the AtariSIOBackend modules. Depending on exactly where I added the delays, and how large I made them (from 30 microseconds to several hundred), things would improve but never be completely smooth.

And then I noticed this little section, where a previous author had already inserted a medium-sized delay:

bool StandardSerialPortBackend::writeComplete()
{  
     SioWorker::usleep(300);
     return writeRawFrame(QByteArray(1, 67));
}
bool StandardSerialPortBackend::writeError()
{
     SioWorker::usleep(300); 
     return writeRawFrame(QByteArray(1, 69));
}

Hmmm. Changing those two instances to SioWorker::usleep(500); instead fixes the .atr stalling/stuttering issue completely, all by itself, with no need whatsoever for any additional delays or changes to the I/O code at all.

 

Now once again, I have no idea how this would impact Linux users; it's basically adding 200 microseconds to the tail-end of these operations, which is just enough to smooth out the reads under OS X, though apparently not needed for *nix in generally. I'm happy to add these changes into separate little conditional blocks if they affect Linux users, but it would be interesting to see if these changes that I've made negatively affect Linux users or make no difference at all.

 

Lastly, I got annoyed by a set of continuing cautions I was getting with regard to line 80 in serialport-unix.cpp. The warnings didn't affect compilation but after thinking about it a lot, the way the code is written doesn't make any sense. Here is what's there by default from AspeQt:

if (!ioctl(mHandle, TIOCMGET, &status) < 0) {

This is Boolean logic-fail at work, it seems to me. How can the result of a Boolean operator be negative? So I added parentheses around the left side of the expression to silent one caution, and changed the comparison value to "1" so the Boolean comparison actually makes sense logically, which silences the second warning I was getting on compilation:

if ((!ioctl(mHandle, TIOCMGET, &status)) < 1) {

Now having said all of this, everything I've been doing has been to fix read operations, not writes. I have done very, very little write testing of this software at all. I suppose I need to open up some productivity apps and beat on it to see if there are problems in that regard.

 

so that first fix was basically making it not poll for command, which takes over CPU totally...

 

the second couple were to lengthen time for some status return, which must have been occuring a bit too early for the atari to register.

 

That last one, I'm not totally sure about. ioctl does NOT return a boolean, it returns an int. operator precedence says that the logical NOT will occur first, so it's doing something weird there. the logical NOT shouldn't be there at all really, you should be able to accomplish whatever needs to be done there without that. if you look a few lines down, you have the same thing almost, but without the relational operator '<'

 

I don't know how casting an int to a boolean works, but regardless, this should be the following instead:

if (ioctl(mHandle, TIOCMGET, &status) < 0) {

that way you have no casting of an int to a boolean. Maybe this is what happens anyway, basically, but this is far easier to understand, and would definitely work. Any other instances such as this should be changed as well, like line 85 of the same file.

 

I have made the changes for line 80 and 85 to serialport-unix.cpp and will probably push sometime tonight or tomorrow.

  • Like 1
Link to comment
Share on other sites

so that first fix was basically making it not poll for command, which takes over CPU totally...

 

Right. Regardless of how that original code works on Linux, on OS X it sucks every cycle it can grab from the CPU core running that thread. It may work differenty for earlier versons of OS X, but since at least 10.9 it's done the same thing. Changing that code to a defininte time makes all the difference in the world. HiassofT suggested 1000 microseconds at first, and then later changed it to 500. That puts a bit more load on the CPU than 1,000 but I don't know if it makes any difference. I will fool around with that tomorrow.

 

the second couple were to lengthen time for some status return, which must have been occuring a bit too early for the atari to register.

 

Yes, those extra couple hundred milliseconds make all the difference. Without the added delay, I might be able to read anywhere from 5 sectors to perhaps even as many as 50 or 60, but inevitably things will choke and get out of sync. This is the same on both an 800XL and 1200XL. I guess for completeness sake I should pull out my 800 which is on the shelf in the other room and test it on earlier hardware. I worried that this was perhaps something with my particular SIO2USB device, but that same device works great with the commercial SIO2OSX software as well as prior AspeQt for Windows software. The problem seems to be in that serialport-unix.cpp code.

 

That last one, I'm not totally sure about. [sNIP]

 

You'd know way more than I do! I've said all along I'm just a hack with a decent set of tools. I know next to nothing about C++ syntax in general, let alone I/O operations. There was no way I could grasp what was going in these blocks specifically without a lot more time to study them - as well as C/C++ in general! My last serious coding was done in college in Pascal to solve differential equations and numerically integrate engineering problems.

 

I have made the changes for line 80 and 85 to serialport-unix.cpp and will probably push sometime tonight or tomorrow.

Great! Thank you for looking further at this part of the code. I'm subscribed to the project on Github so I'll download the .zip when you've made the changes and put together another OS X version to test. In the meantime, I'll experiment with changing that initial QThread::usleep() value and see how it affects things. Perhaps a lower value will help write operations - I haven't tested those much yet but I did see some SIO speed change messages while doing preliminary write tests earlier. On the upside, as things stand now I have yet to find an .atr file that causes a loading problem, and I've tested a few dozen.
Link to comment
Share on other sites

Yes, those extra couple hundred milliseconds make all the difference.

you mean microseconds, it was usleep, right?

 

anyway, r1 has been officially released. I'll make another topic for it. I'd like to have OSX fixes in r2, once you get them all sorted out. I've made an attempt at getting a hackintosh VM, but I've had some issues. Seems my i5 may be a bit too new, so I'll mess with it a bit more tomorrow maybe. Maybe at some point I can really look through the serialport-unix.cpp code and see how everything works, and then try to determine how everything should be done logically, including delays.

Link to comment
Share on other sites

you mean microseconds, it was usleep, right?

Well really, milli-, micro- ... What's a few orders of magnitude? Lol! Yes. I meant microseconds. I'd blame autocorrect but it was just a typo.

 

anyway, r1 has been officially released. I'll make another topic for it. I'd like to have OSX fixes in r2, once you get them all sorted out. I've made an attempt at getting a hackintosh VM, but I've had some issues. Seems my i5 may be a bit too new, so I'll mess with it a bit more tomorrow maybe. Maybe at some point I can really look through the serialport-unix.cpp code and see how everything works, and then try to determine how everything should be done logically, including delays.

Sounds great. I'll grab the latest code tonight and start tweaking to make sure I can get some Mac-specific conditional compilation blocks in there so none of my particular timing changes affect Linux users.

 

 

Sent from my iPhone using Tapatalk

Edited by DrVenkman
Link to comment
Share on other sites

Maybe at some point I can really look through the serialport-unix.cpp code and see how everything works, and then try to determine how everything should be done logically, including delays.

I finally found a little bit of spare time and had a closer look at serialport-unix.cpp - and noticed some odd things:

 

In StandardSerialPortBackend::writeRawFrame (line no 517) the tcdrain block should be at the end of the method (just before the "return true"). This'll make sure that any written bytes will be handed over to the device before the method returns (the OS/tty layer might do some buffering).

 

Getting 100% accurate timing is still tricky, as the serial devices usually have hardware buffers (eg FIFO and transmit holding register in standard 16550 ports). Also transmission might not start immediately. But, fortunately, the timing requirements of the SIO protocol aren't that strict (at least when emulating a SIO device, interfacing to a 1050 requires more effort).

 

Just assume that the device might still be transmitting the byte(s) when writeRawFrame() returns, at 19200 bit/sec this is approx. 520µs per byte. It's quite safe to increase the delay before the complete is sent, eg from 30 to 300 or even 500-1000µs in the worst case this'll slow down transmission a little bit - but that's better than having occasional errors or hangs.

 

The ioctl error handling in serialport-unix looks quite strange and is actually wrong. ioctl usually returns 0 on success and -1 or a negative value on error.

 

This line is wrong:

if (!ioctl(mHandle, TIOCMGET, &status) < 0)
The "!" should be removed - i.e. just check for "< 0"

if (ioctl(mHandle, TIOCMGET, &status) < 0) {

This line is even worse, as it hides a bug:

if (!ioctl(mHandle, TIOCMSET, status)) {
Actually that should be:

if (ioctl(mHandle, TIOCMSET, &status) < 0) {
The second parameter must be "&status", not "status". Otherwise ioctl returns with -1 and sets errno to EFAULT (bad address). This means, the code to drop the DTR and RTS lines has no effect.

 

Fixing this second ioctl requires some testing though as now the state of DTR and RTS might be changed from the previous unaltered default and some SIO2PC adapters might not like that.

 

so long,

 

Hias

  • Like 2
Link to comment
Share on other sites

  • 2 weeks later...

Finally got my Hackintosh laptop up to ML 10.7.2 and gave some of the OS X builds a spin, but since I haven't yet installed the Qt libraries, all the builds complain of a missing printing support library (I can get a screen grab of the error if need be, but it's pretty much the same for all existing OS X builds, even the old AspeQt OS X builds). Is this a normal situation, or something to do with running the Kernel in 32-bit mode (OS X doesn't complain that it can't run the app; it just bombs out when it can't find the library support file)?

Link to comment
Share on other sites

Is this a normal situation, or something to do with running the Kernel in 32-bit mode (OS X doesn't complain that it can't run the app; it just bombs out when it can't find the library support file)?

I don't think 32-bit mode is the problem. I seem to recall reading about that when I first tried building OS X versions of AspeQt last summer. There's an option somewhere in Qt Creator that takes that into account and builds the binary differently. I think. :) I'm not at my Mac right now but I'll look into it tonight after work and see what I can figure out.

 

 

Sent from my iPhone using Tapatalk

  • Like 1
Link to comment
Share on other sites

If you have feature requests or suggestions, let me know.

What I missed with all emulations over the decades until today is a graphics print output. Text print is available only. It would be nice to have the chance to see the print results of programs like print shop, newsroom, mini office, etc. on screen, and having the option to save them as e.g bmp or jpg.

  • Like 1
Link to comment
Share on other sites

Great - many thanks. ;)

Okay you went and made me curious. Looks like you may be correct with your supposition about 32-bit being the problem. I quickly scanned the notes for Qt, and it seems that 64-bit is now the default build configuration for the last couple versions of OS X. I've got more reading to do later, but I should be able to set up a 32-bit deploy configuration tonight and upload a new build for you to try.

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

Okay you went and made me curious. Looks like you may be correct with your supposition about 32-bit being the problem. I quickly scanned the notes for Qt, and it seems that 64-bit is now the default build configuration for the last couple versions of OS X. I've got more reading to do later, but I should be able to set up a 32-bit deploy configuration tonight and upload a new build for you to try.

Heh... so Lion is a little less clear than SL, which simply said it wouldn't run a 64-bit binary in 32-bit mode. Unfortunately the machine has Intel GMA X3100 graphics and there are no 64-bit drivers, hence running the kernel in 32-bit mode. I'm surprised it even touches the 64-bit build, unless the app space is still 64-bit. No idea. I understood it to be a massive ball-ache to compile 32-bit Qt stuff for OS X these days, possibly involving a complete re-build of Qt itself. I'll be pleased as punch if this is not the case.

 

Anyway: thanks in advance. Keen to do some testing.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...