VAPI ATX Format available ?

[phaeron]

The timing is there. It is the "Sector position" field in the sector list. But I see I forgot to include the units. The unit is in 8 microseconds (one FM bit time) at nominal RPM.

 

As you can see, the sector position is specified as an angular/radial dimension (a given time period at a fixed rotational speed, gives you an angle), which is not affected by the speed of the recording or imaging drives. Only that instead of being specified as degress or radians (which would have been akward), the unit is the angle covered by one full FM bit cell when operating at nominal conditions.

 

If the spec were specified directly as an angular unit, that would be perfect, as that's what I actually need. Unfortunately, even knowing that it is in "one full FM bit cell when operating at nominal conditions" isn't a good specification. FM is an encoding method and has no inherent rate spec of its own -- all that is required is that the recording medium can support the signal and that the recording and reading drives are sufficiently within tolerance of each other.

 

To put it another way, if I were reading the spec, how would I know what is "nominal"? Is it a 288.000 RPM drive with a 1.0000MHz clock frequency? You can't derive these numbers from the spec. I assume you mean based on the Atari 810 drive, but then that raises additional questions. For instance, if it turns out that Atari used something like a 0.998MHz crystal in practice, is that what I should use as the basis for determining the angular unit?

 

I realize you don't like the DLL approach, but that's the method that bring the most compatibility, far and beyond publishing anything else.

 

What this effectively means is that you have a reference implementation which everyone has to reverse engineer in order to write their own implementation with the same level of compatibility, which is not good for a spec. It's standard practice for specifications to have at least an "informative" section which includes information that doesn't strictly need to be part of the spec but contains useful information on how not to screw it up, which is important to avoid incompatibilities.

 

I'm not saying that you have to write a full spec for complete emulation of an 810 floppy drive, or completely document the inner workings of your DLL. What I'm saying is that for people to use your spec, it's helpful to give at least basic drive parameters as a model for validation. This is especially important since, as I noted, some of those parameters are NOT documented by Atari or easy to determine. Just describing a validation reference like "288 RPM, 40ms step, 1MHz bit rate, 10ms head settling time, 19040 baud SIO transmission rate" would be very helpful for anyone trying to use this spec.

 

By the way -- in case I gave the wrong impression, I appreciate the work you've done with VAPI in establishing a useful and faithful way to image Atari disks and I know that you're time constrained. Writing a spec is actually difficult and a lot of work. If you don't mind and I get some time, I might take a shot at augmenting your spec and trying to fill in some of the gaps.

[ijor]

I think I should better follow up on PM.

[Kr0tki]

Please answer publicly, I am interested in the technical issues standing behind creating the VSPI specification.

[ijor]

Hi Kr0tki,

 

Please answer publicly, I am interested in the technical issues standing behind creating the VAPI specification.

 

It wasn't too technical, that's why I thought better to follow-up by PM

 

The main technical issue is that "one FM bit cell" means 8 microseconds, as actually mentioned explicitely in the previous post. I thought it was obvious that nominal conditions refers to a standard Atari 8-bit drive, 810 or 1050.

 

The main idea is that when you are emulating an Atari drive, you'll usually do it at nominal conditions. Then you can consider the unit as plain time instead of an angle, which is obviously much more convenient.

[Kr0tki]

OK, I understand.

 

My concern is, that if the VAPI format's purpose is "to exactly describe physical structure of a disk", then it should contain only data that is needed to fill this purpose and not anything more. In the current situation, you got to have some external information (810's speed characteristics) in order to use a VAPI image. Since that external information is "not a few numbers", and it varies from drive to drive, implementing a proper VAPI support starts to be a bit of a burden.

 

VAPI format relying on characteristics of a 810 drive has the following consequences:

- The 810's characteristics are different for each individual drive, and some parts of the characteristics aren't documented and have to be determined experimentally. The "nominal" characteristics chosen as a basis for VAPI, had to be decided on pretty much arbitrarily - while such decision would not be needed at all if the VAPI format was not based on an 810 as it is;

- If an emulator author wants to emulate other drive models, he would need to devise some rather complicated conversion routines in order to allow VAPI images to be used with other drives;

- In order to create a VAPI image using a disk drive that has characteristics even slightly different than the nominal ones, a similar conversion process is needed;

- If in the future some new important characteristics of the 810's speed are found (or measured more exactly), and there exist some disks that rely on those characteristics, then the VAPI format would need to be extended to incorporate such characteristics. If the VAPI format was not based on an 810, it would be saved from changing in such cases;

 

When the 810's characteristics are not in the spec, it poses another problems:

- The VAPI specification is not enough to retrieve disk's physical structure from a VAPI image;

- In order to implement a VAPI support, one has to contact the author and ask for missing data.

 

The thing I'm interested with, what were the technical reasons that prevented you from basing the VAPI format directly on some angular unit and not on specifics of a particular disk drive?

[ijor]

OK, I understand.

 

Hmm, I'm not sure you really do. And it is likely my fault. If two experts like Phaeron and you misunderstand me, then must be me failing to explain and describe things properly. I am saying this honestly, not ironically.

 

While replying to your post, I am reordering some of your paragraphs. Hope you don't mind, I am doing this because I think one thing lead to the other and then my reply, hopefully, would be clearer.

 

The thing I'm interested with, what were the technical reasons that prevented you from basing the VAPI format directly on some angular unit and not on specifics of a particular disk drive?

 

But it is an angular unit. It is 0.013824 degrees (assuming I did the math correctly). In other words, if one sector in the VAPI image has sector position 0 (zero), another sector that is exactly on the opposite position (180 degrees apart) would have a sector position 13,021 (180 / 0.013824, rounded).

 

The math btw, is: (360 * RPM * FM_BIT_TIME_IN_MICROSECS) / (60 * 1000*1000) = (360 * 288 * 8 ) / (60,000,000) = 0.013824

 

I guess I should have said something like this all the time:

 

The sector position field is in angular units, each unit is 0.013824 degrees. This is the same, and not by chance, as the angle covered by one full bit cell at FM on an Atari 810 or 1050 drive operating at nominal conditions. Nominal conditions means that the FM bit time is 8 microseconds, and the rotation speed is 288 RPM.

 

Why I didn't say that before? Why I never actually use it myself like that (and then I had to compute it for the purpose of this answer)? Well, I don't use it like that because it would be akward and cumbersome to use. And I thought (may be I was wrong) that the way I said it, it was obviously equivalent (more about that below). It honestly didn't cross my mind that it would produce such misunderstanding.

 

My concern is, that if the VAPI format's purpose is "to exactly describe physical structure of a disk", then it should contain only data that is needed to fill this purpose and not anything more. In the current situation, you got to have some external information (810's speed characteristics) in order to use a VAPI image. Since that external information is "not a few numbers", and it varies from drive to drive, implementing a proper VAPI support starts to be a bit of a burden.

 

I think we are mixing two very different things. One thing is which information is needed to physically describe the disk image. Another one is what other information you need to actually use the image (at least under emulation or for a SIO2PC/APE application).

 

The information we need for the physical description is, in the worst case, which ones are the 810 and 1050 nominal conditions. More precisely, we need to known that the FM bit time is 8 microseconds, and that the rotational speed is 288 RPM. No other 810 (or 1050) characteristic is required for the purpose of physically describing the image. These two values are enough to reach the value in degrees I mentioned above, and they are enough to write back the image on a new physical disk for that matter. I regret that I was silly enough to not be more explicit in my previous posts, let alone in the preliminary specs document. Once being explicit as I was above, then actually no "nominal condition" feature need to be assumed for this purpose. You can forget what I said about "nominal conditions", and consider them just as a shortcut for the value in degrees. Again, I thought all the time that this was obvious.

 

Now, is that enough for emulating a drive "implementing a proper VAPI support"? Of course not. You need a lot more of details about the exact behavior of an Atari drive. You need a lot more information about Atari 8-bit in general for that matter. This includes such things as how the drive firmware handles the stepper, what is the exact step rate algorythm, and a load more. But as I was saying, this is not directly related to VAPI, and it can't be a formal part of the specifications.

 

I actually understand that Phaeron agree with me here. What he claims is that, even when not strictly part of the specs, they are very valuable and hard to find information, that it would be good and important if they were mentioned one way or the other. I assume from what he is saying, for instance, that Phaeron spent a lot time until finding out and realizing which is the effective SIO transmission rate on an 810 drive. That is important for the actual timing on some picky releases.

 

Part of my PM to him was to actually apologize about that. But as I was saying, these ones aren't simple numbers and values. It is a more or less complex behavior, and that it depends on the drive type. So all the time I wanted to write an article commenting and elaborating these issues, can't be done mentioning just a few numbers. I knew all the time that accurately emulating a drive wasn't easy, but I didn't consider that mentioning a few values as Phaeron did above, would make the difference. I was possibly wrong, but what I think is that it requires a good and detailed understanding of the 810/1050 firmware and hardware regardless. I suspect that Phaeron just had bad luck (or good luck, depending on the POV) because he tested certain VAPI images. Because there are other releases that don't depend exactly on this type of timing, but they are possibly more difficult to emulate; and I suspect he didn't test them.

 

However, and once again, none of those values are part of the "nominal conditions" we were talking about earlier. None of these values and info is required for the purpose of describing the disk image and writing back the images to a physical disk. They are all valuable information that is useful (even required, if you want) for the purpose of implementing an accurate Atari 8-bit drive emulation. The "only" relation with VAPI, is that an accurate and precise emulation is required for many copy protections. And once again, even some computer (not drive related) aspects, that normally affect copy protections only, have to be emulated accurately. For instance, I mentioned long ago the need to properly emulate the 400/800 behavior when accessing the C000-CFFF memory space.

 

VAPI format relying on characteristics of a 810 drive has the following consequences:

- The 810's characteristics are different for each individual drive, and some parts of the characteristics aren't documented and have to be determined experimentally. The "nominal" characteristics chosen as a basis for VAPI, had to be decided on pretty much arbitrarily - while such decision would not be needed at all if the VAPI format was not based on an 810 as it is;

 

I don't agree with this, and hopefully after all the elaboration above we come to agree. The nominal characteristics I was talking about (FM bit time and RPM) are nothing but arbitrarily, they are fully and precisely documented and certainly they aren't different across individual units (actual conditions would of course vary, but not nominal ones). And again, the "nominal conditions" expression was used just as a shortcut. I am assuming that a couple of things I said were a bit misleading here.

 

I said that some values and the exact behavior depends on the drive type. That means, for instance, that the exact effective transmission rate is different between a stock 1050 and a Happy enhanced one. But regardless, this data is not one of the nominal conditions we were talking about.

 

I also said that a "reasonable average" works for most, but not all cases. This is again, related to the difference across drive types mentioned above. And once again, they don't apply for the nominal characteristics needed. As I said, the correct comprehensive solution is to emulate multiple drive types explicitely, and let the user choose among them (same way we now choose the exact computer model), or do it somehow automatically.

 

- If an emulator author wants to emulate other drive models, he would need to devise some rather complicated conversion routines in order to allow VAPI images to be used with other drives;

 

I don't think you would require any complicated conversion at all. Again, I hope this is now clear, but otherwise please let me know.

 

- In order to create a VAPI image using a disk drive that has characteristics even slightly different than the nominal ones, a similar conversion process is needed;

 

I do this all the time. I produce VAPI images on standard PC drives, some operating at 300 RPM, other ones at 360 RPM. All that you need is to apply a simple scaling, in the worst case.

 

- If in the future some new important characteristics of the 810's speed are found (or measured more exactly), and there exist some disks that rely on those characteristics, then the VAPI format would need to be extended to incorporate such characteristics. If the VAPI format was not based on an 810, it would be saved from changing in such cases;

 

Ditto. Again, i hope it is now clear that the VAPI format doesn't depend on an 810 characteristic, not in the sense you were thinking. It is possible that some disks depend on very specific drive characteristics that we didn't measure yet (say, some timing of a third party drive, or some specific firmware revision). This might require enhancement of the emulation implementation, not any changes to the format or the disk image.

 

 

And just in case somebody asks which part of the sector is the one that determines the position, the short answer is that it doesn't matter. What matters and is relevant is the relative position (angular distance) between sectors. Most Atari drives don't have an index hole sensor, and then the absolute position doesn't matter. Furthermore, many disks were recorded in the flippy side, so that the index hole is not available anyway. This is true not only when actually using the disk, but also when imaging the disk in drives likes a 1050 enhanced one. However, when imaging the disk and the index hole is available, then we conventionally use the distance between the start of the index and the start of the sector header.

[phaeron]

I did a pass on revising the spec, based on the current information that I know.

 

Three important changes relative to the 0.1 spec:

• Sector angular position is specified in a direct, drive-independent unit.
  
• There was an error in the type value specified for sector lists -- it is actually $01, not $00.
  
• Weak bit information is documented. It's probably quite wrong, but it seems to work so far....
  

 

I updated Altirra's ATX parser to match this spec and verified that every ATX disk image that I have still boots correctly.

AtxSpecsPrelim-0.2.txt