Altirra 3.10 released

[phaeron]

Nuclear time. This ROM just sets the background color to $14, $44, or $74 depending on whether the reset vector, coldstart vector, or warmstart vector are invoked, and then halts with interrupts disabled. We should at least get a color.

 

kernel.rom

[Faicuai]

Nuclear time. This ROM just sets the background color to $14, $44, or $74 depending on whether the reset vector, coldstart vector, or warmstart vector are invoked, and then halts with interrupts disabled. We should at least get a color.

 

 

This is completely ABSURD... What the hell is going on here? Still stuck on the black-screen... Proceeded again to cycle ALL BIOS options, INCLUDING those OUTSIDE of Colleen profile, and then coming back to it, just in case some screw is lose somewhere in there... but nothing (official OEM A/B loads continue to work perfectly, though).

 

HOWEVER, I have an idea (let's grab the bull by the horns): The way I see it, we are either LANDING or NOT landing on the OS (one of the two). If we are, indeed, landing on the OS, what we also don't know is WHERE... In that context, would it be possible to fill the entire rom (except your code) with NOPs, until the last 3 bytes, in which we hard code a jump straight to your code (or something even tinier) so we know that we are actually landing on that ROM segment?

 

Let me know if this make sense, and I will try to make it happen, here...

Edited May 21, 2019 by Faicuai

[phaeron]

 

This is completely ABSURD... What the hell is going on here? Still stuck on the black-screen... Proceeded again to cycle ALL BIOS options, INCLUDING those OUTSIDE of Colleen profile, and then coming back to it, just in case some screw is lose somewhere in there... but nothing (official OEM A/B loads continue to work perfectly, though).

 

HOWEVER, I have an idea (let's grab the bull by the horns): The way I see it, we are either LANDING or NOT landing on the OS (one of the two). If we are, indeed, landing on the OS, what we also don't know is WHERE... In that context, would it be possible to fill the entire rom (except your code) with NOPs, until the last 3 bytes, in which we hard code a jump straight to your code (or something even tinier) so we know that we are actually landing on that ROM segment?

 

Let me know if this make sense, and I will try to make it happen, here...

 

Could try that, though a NOP sled won't work if the CPU is landing outside of the ROM. When I looked at the latest BIOS, though, it looked like it would only jump to either ($FFFC) or WARMSV. The current code is pretty minimal, it's LDA #imm / STA COLBK to set the color, SEI + LDA #0 + STA NMIEN to kill interrupts, and JMP * to lock.

[flashjazzcat]

Before blaming the OS, it's probably worth verifying that any 800 OS flashed to one of the Colleen OS slots works. To that end, I've flashed Avery's OS to Colleen slot 4, found it not to work, and dumped the slot back out using UFLASH. The dump differs from what was flashed starting at offset $1800, so I then dumped the entire ROM back out to eliminate issues with the OS dumping mechanism, and the same error exists there. So there's almost certainly nothing wrong with Avery's OS; the fault appears to lie with UFLASH's Colleen OS flashing code. This may have become broken recently or may have been broken for years... I have no idea, since no-one reported an issue since 2014 when the program was first written. I guess not much flashing of alternative 800 operating systems has taken place.

 

I'll bake Avery's OS into the ROM with a hex editor and test it just to be sure, of course, but I think he can stop wasting any more time on this.

[flashjazzcat]

And yes: Avery's OS works perfectly when properly situated in the ROM slot.

 

Fixing Incognito related issues is always fun, since it's real-hardware seat of the pants stuff only. The fact some parts of UFLASH seem impenetrably complex five years after the event does not help.

[flashjazzcat]

*Fixed.

 

New issue is that the loader can't see the HDD when running the Altirra 800 kernel (works OK with OS-A/B).

 

@Faicuai: here's an updated UFLASH:

 

uflash.xex

 

Loader gets stuck forever in the 'Waiting for media...' phase with this OS, which is odd since all it's doing is putting the CF card in 8-bit mode and issuing an Identify Device command.

 

EDIT: Loader issue fixed. Incognito build was blithely setting the PBI-present flag without actually testing for a PBI ROM. The Altirra OS must have got hung up on an SIO call with a spurious mask in $D1FF in a manner which OS-A/B does not. So that's two bugs fixed today.

Edited May 21, 2019 by flashjazzcat

[pseudografx]

I would really welcome if Altirra could output screenshots with indexed palette, not because of the size reduction, but because it makes it easier for me to work with the palette in my graphics editor.

[Faicuai]

*Fixed.

 

New issue is that the loader can't see the HDD when running the Altirra 800 kernel (works OK with OS-A/B).

 

@Faicuai: here's an updated UFLASH:

 

uflash.xex

 

Loader gets stuck forever in the 'Waiting for media...' phase with this OS, which is odd since all it's doing is putting the CF card in 8-bit mode and issuing an Identify Device command.

 

EDIT: Loader issue fixed. Incognito build was blithely setting the PBI-present flag without actually testing for a PBI ROM. The Altirra OS must have got hung up on an SIO call with a spurious mask in $D1FF in a manner which OS-A/B does not. So that's two bugs fixed today.

 

Wow!!! Almost a miracle we caught this one!!

 

A HUGE thanks for your masterful CSI / forensic work... no one here could have fixed this problem without your intervention, though (in fact, we may had gone mad, already)... The very last thing I thought could have been having issues, though... So what's up with all the "Flashing / Verifying/ OK" messages we got on U-Flash? Was it all a dream, or did it actually happen... on a multiverse???

 

I do wonder what has happened all this time... how come no one else ever reported this issue? I actually thought it 3.1 was just an experiment and would be corrected later... But when I saw it re-emerging with Kernel 3.2 load, I knew something was wrong...

 

Now, the SIDE-LOADER remains stuck (with 3.2 Kernel), though... I guess it will have to wait until the second fix (BIOS-level) emerges.

 

I am now testing Avery's 3.2 800-Kernel, and putting it through its paces... I know exactly what to bait it with... Will get back here, later today, on a better California-time.

 

Cheers!

Edited May 21, 2019 by Faicuai

[flashjazzcat]

Wow!!! Almost a miracle we caught this one!!

It's most fortuitous, although since the bug went completely unnoticed for five years (and I now believe it existed for at least four of those years), it does not seem to have been critically debilitating for the majority of users.

 

So what's up with all the "Flashing / Verifying/ OK" messages we got on U-Flash? Was it all a dream, or did it actually happen... on a multiverse???

The error was caused by the 4K block count being one entry short. This caused the flash process and the verification process alike to stop after processing the first 8K of the 12K ROM slot. Since the 800 OS slots are preceded by 2K of padding, this resulted in the wrong data appearing in the last 4K of the 10K ROM (as observed when comparing the dump with the original kernel ROM).

 

Ironically enough, the 4K block count was temporarily reduced (to 2) to allow proper calculation of the progress bar steps, but never bumped back up to 3 prior to the flashing phase. Owing to the incredibly fiddly nature of the Incognito ROM layout (800 operating systems in 12K padded blocks interleaved with the 4K chunks of the 16K loader ROM, etc), special discreet file reading functions are required, and one of them contained a bug.

 

While fixing this, I also took the opportunity to fix a slight overshoot of the file read progress bar when loading an 800 OS.

 

I do wonder what has happened all this time... how come no one else ever reported this issue?

Either because no-one ever flashed an 800 OS before (possible, since I've never flashed anything but the existing content to those slots during testing, which is exactly how this bug was overlooked by me), or - as I've pointed out elsewhere - someone noticed the issue already but didn't bother to tell me or anyone else about it.

 

Now, the SIDE-LOADER remains stuck (with 3.2 Kernel), though... I guess it will have to wait until the second fix (BIOS-level) emerges.

Loader fix, which prevents spurious calls to the SIO when there is no PBI BIOS available. I'm still idly curious to know why SIO calls with non-standard command codes ('M', etc) caused an endless spin cycle with the Altirra OS but no problem whatsoever with OS-A/B. But the fault surely lay with the loader.

[_The Doctor__]

thanks for explanations, it makes for an interesting read and lends itself nicely to help us follow along and better understand. I appreciate this greatly.

[Faicuai]

Could try that, though a NOP sled won't work if the CPU is landing outside of the ROM. When I looked at the latest BIOS, though, it looked like it would only jump to either ($FFFC) or WARMSV. The current code is pretty minimal, it's LDA #imm / STA COLBK to set the color, SEI + LDA #0 + STA NMIEN to kill interrupts, and JMP * to lock.

Whoa!! Well, we finally got this one working on real HW, with FJC's direct support !!! (absolutely key, considering that the issues ended up being non related to Altirra's 800 kernel)

 

Ever since I tested the XL-based loads, I not only noticed the peculiar FPP package, but also some other (nice) things under the hood. Since then, I could only wonder if some of that cool stuff found its way into the 800-kernel version...

 

Well, so everyone here can see and judge what these OS-loads are all about (and not just dummy re-implementations), here is a before (left, OS/B) and right (Altirra 800 Kernel 3.2) running some basic tests on SI 2.24:

 

 

ALL of OS basic operations (text output, scrolling, dot and line drawing, you name it) have been subject to substantial performance boosts (even visible with the naked eye)... But what is REALLY impressive, which caught my eye during XL-version tests (and now present on the 800-load) was the draw/fill improvement... Seeing it action is remarkable, considering the rom-space constraints.... KUDOS!!!

 

The FPP load of the rom also performs easily in the 2.0 to 3.5x speed range, across the board, while performing computations x100 to x1000 times more precisely (!!!) as visible in some classical basic benchmarks!

 

Now, having said this, I also went for a test drive on some "must-pass / acid" titles that puke out unless OS/B is alive and well. Here's what I found:

 

1. APX Block Buster (for OS/B): corrupted Gr.0 screen (bottom) during load, full crash after loading, with "ringer of death" (I swear is the loudest bell I've ever heard coming out of the 800 !!! :-)

2. Synapse's Nautilus: Fails to display background rainbow, yellow edge and Full crash. Exactly like XL.

3. Attack on EP-CYG-4: corrupted character-set, hardly playable.

4. Space Cowboy: great isometric (?) perspective, and to my surprise loads and work, except for timings which seem tied to original OS/B + non-accelerated FPP set.

 

NOTE:

1. Dual-cart (LEFT and RIGHT) seems booting fine. Same easy-to-fool detection / booting as OS/B, but seems to boot everything nicely, so far.

 

In any case, very promising, and will leave it flashed on slot #1, and I am sure these inks will progressively be figured out.

Edited May 22, 2019 by Faicuai

[phaeron]

ALL of OS basic operations (text output, scrolling, dot and line drawing, you name it) have been subject to substantial performance boosts (even visible with the naked eye)... But what is REALLY impressive, which caught my eye during XL-version tests (and now present on the 800-load) was the draw/fill improvement... Seeing it action is remarkable, considering the rom-space constraints.... KUDOS!!!

 

The FPP load of the rom also performs easily in the 2.0 to 3.5x speed range, across the board, while performing computations x100 to x1000 times more precisely (!!!) as visible in some classical basic benchmarks!

Something is weird with your SysInfo timings, they are about 30% higher than what I have been seeing. AltirraOS is generally faster, but not that much faster. Are you seeing 99-100% with the XL/XE OS?

 

Now, having said this, I also went for a test drive on some "must-pass / acid" titles that puke out unless OS/B is alive and well. Here's what I found:

 

1. APX Block Buster (for OS/B): corrupted Gr.0 screen (bottom) during load, full crash after loading, with "ringer of death" (I swear is the loudest bell I've ever heard coming out of the 800 !!! :-)

2. Synapse's Nautilus: Fails to display background rainbow, yellow edge and Full crash. Exactly like XL.

3. Attack on EP-CYG-4: corrupted character-set, hardly playable.

4. Space Cowboy: great isometric (?) perspective, and to my surprise loads and work, except for timings which seem tied to original OS/B + non-accelerated FPP set.

 

In any case, very promising, and will leave it flashed on slot #1, and I am sure these inks will progressively be figured out.

 

All of these programs are either using internal addresses within the OS or have bugs. These aren't bugs in AltirraOS and I can't fix them because it would impose unreasonably restrictive code placement or require AltirraOS to be too dangerously similar to the Atari OS.

APX Block Buster: Illegal jump into OS CIO at $E4C4. $E4C4 is the actual start of CIO pointed to by the JMP at CIOV in OS-B, so this saves a whopping three cycles.

Synapse Nautilus: Broken timer 1 routine. It pushes A/X/Y but only pulls Y/X, then returns with RTI instead of RTS. This causes the CPU to execute the high byte of an internal subroutine address as an instruction.

Attack on EP-CYG-4: Hardcoded reference to ATASCII-INTERNAL table at $FEF6. AltirraOS uses a different conversion algorithm and doesn't even have this table.

Space Cowboy: Illegal jump into OS at $F369 for, of all things, an INC $09 / RTS sequence. Unnecessary, as $09 isn't even used before being overwritten.

 

In general, any program marked as OS-B only in collections won't work on AltirraOS for the same reason that they don't work on the XL/XE OS.

[phaeron]

Loader gets stuck forever in the 'Waiting for media...' phase with this OS, which is odd since all it's doing is putting the CF card in 8-bit mode and issuing an Identify Device command.

 

EDIT: Loader issue fixed. Incognito build was blithely setting the PBI-present flag without actually testing for a PBI ROM. The Altirra OS must have got hung up on an SIO call with a spurious mask in $D1FF in a manner which OS-A/B does not. So that's two bugs fixed today.

 

This is interesting. AltirraOS/800 doesn't have PBI support, of course, because there is no parallel bus interface on the 800. The setup code, CIO/SIO hooks, and interrupt support are all compiled out with _KERNEL_PBI_SUPPORT=0. If Incognito does support PBI emulation in Colleen mode, then there would be an issue as an 800 OS wouldn't clear $D1FF the way that a PBI-capable OS does. However, in that case you would also have problems with OS-B.

 

SIOV largely doesn't interpret the commands that are sent, as it relies on DSTATS to select the frame transfer mode. The only special handling is for DDEVIC=$60, which activates the special cassette tape support. Unhandled commands are extremely common, it's basically impossible to boot DOS unless command timeouts are handled properly.

 

Let me know if there's something I can track down on my side.

[Faicuai]

Something is weird with your SysInfo timings, they are about 30% higher than what I have been seeing. AltirraOS is generally faster, but not that much faster. Are you seeing 99-100% with the XL/XE OS?

 

They are fine. And they are so, because Antic is turned off (Option on SysInfo config menu). This allows measuring code-to-code raw performance. The OS/B sample is baselined in the exact same way (so it is apples-to-apples).

 

 

All of these programs are either using internal addresses within the OS or have bugs. These aren't bugs in AltirraOS and I can't fix them because it would impose unreasonably restrictive code placement or require AltirraOS to be too dangerously similar to the Atari OS.

Thanks for the DETAILED debugging (very much appreciated!) Just wondering how far could the Kernel-800 go as a replacement of OS/B. It is good to know where the limits are, for sure. The thing is that an 800-targeted OS makes sense, for the most part, for running OS/B titles (where the majority of the 800-software actually resides)... but if the kernel deviates quite a bit (which I know it is part of the idea), well, a potentially large (?) # of titles may not work either with the alternate kernel. I will give it a longer spin, and see how far it goes (very interesting under the hood, though...)

 

Cheers!

[flashjazzcat]

If Incognito does support PBI emulation in Colleen mode, then there would be an issue as an 800 OS wouldn't clear $D1FF the way that a PBI-capable OS does. However, in that case you would also have problems with OS-B.

Exactly so, and I doubt the spurious write to $D1FF is the actual cause of the issue. A similar bug in an older version of the stand-alone SIDE loader caused issues with the XL/XE OS, since the SIO handler will still jump through $D805 regardless of the absence of a handler with valid signature bytes if $D1FF is manually manipulated, even if the math pack is still there (for the obvious reason that the signature check is only performed when setting up PDVMSK). The reason for direct manipulation of $D1FF in the case of the loader is to ensure that only a specific PBI device (where more than one is present which may support the same command set) responds to the special SIO commands.

 

SIOV largely doesn't interpret the commands that are sent, as it relies on DSTATS to select the frame transfer mode. The only special handling is for DDEVIC=$60, which activates the special cassette tape support. Unhandled commands are extremely common, it's basically impossible to boot DOS unless command timeouts are handled properly.

 

Let me know if there's something I can track down on my side.

I'll try to force the endless loop manually and narrow down exactly where it's occurring.

Edited May 22, 2019 by flashjazzcat

[flashjazzcat]

I created a build of the Incognito SIDE loader (with the bug that causes PBI-dependent SIO commands to be issued) which runs off a SIDE2 and ran this with the 800 kernel. Here's one place where it hangs:

 

 

Here's the command frame:

 

 

I'm assuming the polling loop at $EB2E relies on the OS VBI in order to timeout, and since the loader completely replaces the VBI with its own interrupt handler (which services only the RTC and the countdown timers, since no serial IO is ever explicitly requested), this never happens. Owing to the fact the OS VBI isn't running and this was caused by a bug in the loader anyway, I doubt it's worth worrying about.

Edited May 22, 2019 by flashjazzcat

[phaeron]

Replacing the VBI handler is fine as long as OS timer #1 is serviced (CDTMV1/CDTMA1). The OS Manual declares this timer as being used by SIO for timeouts and it should work the same way on AltirraOS as for OS-B. It's also unskippable since it is serviced in VBI stage 1. What are you setting DTIMLO to on the SIO request?

[flashjazzcat]

The timeout value is as per the screenshot of the command frame ($06), but when I looked at CDTMV1 again, aside from being cleared on a cold power-up, it isn't set at all in this scenario before we enter the endless polling loop. Of course the VBI will only jump through the vector if the timer transitions from non-zero to zero. So presumably something about this particular command frame is causing the OS not to bother initialising the timer?

 

Anyway: I manually set the LSB of CDTMV1 to some non-zero value and set a breakpoint on my VBI, which did duly jump through CDTMA1. However, the code at $EC21 just writes 0 to TIMFLG ($0317) and returns, which doesn't break the deadlock.

Edited May 23, 2019 by flashjazzcat

[Faicuai]

(...) but when I looked at CDTMV1 again, aside from being cleared on a cold power-up, it isn't set at all in this scenario before we enter the endless polling loop. (…)

 

 

Could this then be the answer as to why [OS/B + current SIDE loader] do not fail with in the presence of PBI-dependent SIO-command salvos?

 

Because that's where the solution should reside, imho...

[flashjazzcat]

Yes: that would be the obvious inference based on what I have said, but since Avery wrote the OS, he'll be able to tell us for sure.

[Faicuai]

Yes: that would be the obvious inference based on what I have said, but since Avery wrote the OS, he'll be able to tell us for sure.

 

Mhmm… but does Incognito support (or not) PBI in Colleen-mode? (could not draw a firm conclusion reading the analysis drawn up to this point...)

[flashjazzcat]

 

Mhmm… but does Incognito support (or not) PBI in Colleen-mode? (could not draw a firm conclusion reading the analysis drawn up to this point...)

 

Almost certainly not, but it doesn't matter since the corner-case issue is in evidence even when no Incognito is present and a standard 800 is emulated.

[JohnPolka]

 

There is room for both. Use Altirra as a developer's tool. When you have real hardware you can run RespeQt on your Windows box. You can use an Ultimate Cart... The list goes on.

 

 

I did most of my recent software development on Altirra, because the development process is easier/faster. However, when I worked on my the terminal emulation program for BBS Express Pro!, I discovered that I really need to use native hardware to properly test it. For example, the term program needs to work with a real Atari 850, P:R: Connection, MIO, as well as, the emulations in Altirra. I also tested it with a modem and with a Lantronix UDS-10 device. I found that not all of the idiosyncracies of the real hardware were being captured by the emulator.

 

For high level programming, Altirra is great. For low/system level programming, one should at least make sure their program works on real hardware. And if it doesn't, then they'll have to use real hardware in their development.

 

-JP

Edited May 23, 2019 by JohnPolka

[flashjazzcat]

It doesn't matter how good the emulation is; software is not properly tested until it has been tested on real hardware. But if the target hardware is emulated, I see no problem with getting the software working in the emulator before testing it on the actual hardware. It's when the second step is entirely omitted that problems ensue.

 

Of course, sometimes the target hardware (such as Incognito) isn't emulated at all by any emulator, and then one has no choice but to develop exclusively using actual equipment. But since Altirra and RespeQt can both work with 'virtual folders', it's easy enough to run both at the same time, with both accessing the same assembly target folder.

Edited May 23, 2019 by flashjazzcat

[_The Doctor__]

To be sure, every knows my thoughts when it comes to real hardware. I do however believe that if we didn't have devs using Altirra much of what we have wouldn't exist. Some our best talented people don't have the Atari stuff set up anymore (wife life space kids)... They do the best they can using the tools that let them continue with their passion. It's up to people who have the time and space with real machines/devices to help them out and get the issues tackled. This could be dev partners or the general public. Honestly it's not far from bitd when code was written on an S-100 or other machine (yes big time shared machine too), compiled and checked out sometimes by others. It's a partnership that is gratifying and amazing if you think about it. People in different physical locations all around the world making games/applications/ and even devices with and without the real physical machines. Amazing stuff.