Jump to content
Irgendwer

Do we need a new standard for disk/cart images?

Do we need a new standard for disk/cart images?  

45 members have voted

  1. 1. ATR/CAR images don't cover environmental needs. What about a new format?

    • I would appreciate and even actively support it.
    • I would appreciate that.
    • I don't mind.
    • I doubt, that this is a good idea.
    • No! Not another file format!


Recommended Posts

Torn between "No! Not another file format" and "I would appreciate and even actively support it". If any new format had a 512 byte header, I'd sure appreciate it. That would greatly simplify disk image handling in things like Ultimate 1MB PBI, and even allow for performance improvements (since disk image sectors would align with physical sectors).

Share this post


Link to post
Share on other sites

Yes - but needs to be downward compatible.

 

To have environmental parameters that an emulator can use - at the minimum might eliminate a whole bunch of people complaining about things not working.

 

But once images get out in the wild, it'd probably take years for everyone to get up to date.

Share this post


Link to post
Share on other sites

No, not another file format.

 

What about existing hard and software? How do you handle multi-disk images? What about software requiring both a CAR and an ATR? Multiple, stacked carts?

 

If I understand you correctly, this is all about adding some metadata to help emulator users and (maybe?) library software to help structuring your software archive.

 

Just store the metadata in a separate file, describing the required system parameters (PAL/NTSC, memory etc), which images should be loaded into which drives, other typical metadata stuff (release year, version, author(s), ...) and so on and ZIP everything up.

 

Then convince emulator authors to support your metadata file format (optionally with direct support for opening the ZIP file) and you are done. No need for yet another disk or cartridge image format.

 

BTW: This is basically the same approach as the JAR (java archive) format - a ZIP with all the java classes and resources plus a manifest file.

 

so long,

 

Hias

  • Like 2

Share this post


Link to post
Share on other sites

Bundle or seperate descriptor file has it's merits but probably not real useful on real machines using SIO2SD and the like. Although generally real hardware you're on a fixed configuration and such metadata isn't really vital anyway.

 

But for me, if it can be a seamless addition that doesn't break anything then why not?

Share this post


Link to post
Share on other sites

I see this as a way to wrap existing formats like XEX, ROM, ATR, CAS, etc. Maybe look at SAP and SID/PSID formats for inspiration? Another possibility is to make a separate metadata file. That way you still have the actual file in an established format.

 

Edit: Just realized I repeated what was already said. Anyhow I lean towards a separate file.

Edited by Xuel

Share this post


Link to post
Share on other sites

I'm also against changing the existing container formats (ATR/CAR/XEX). They fully do what they were intended to do and are widely supported.
If we need additional information, it should be added as separate file. Just like Altirra already today automatically looks up the "example.atdbg" "example.lst", "example.lab" files if you start "example.xex".

This file (suggestion ".ati") should be in an extensible text format with well defined encoding (e.g properties/xml) and should be able to optionally include:

- release information (producer, title, year, authors, ...)

- documentation (in multiple languages)

- environment requirements (PAL/NTSC, ROMs, RAM sizes etc, as "required" or "supported")

 

It also gives you the simple choice if you want to run the executable in the "current" environment (click on the ATR) or in the required envionment (click on the ATI).

This would be a compatible, optional extension that might also be valueable on the real machine. In that sense, I vote for "Yes".

 

 

post-17404-0-23284700-1395601641_thumb.png

Share this post


Link to post
Share on other sites

No to extending the existing formats. For one thing, they all have different levels of extensibility: ATR is a fixed size header format, CAR is a chunk-based format, and XEX has no extensibility at all since it is the raw DOS executable format. Second, from a layering standpoint this doesn't belong in the image formats. It would mean that the disk/tape/EXE parser would have to be able to reconfigure global system parameters above it. Ick. We do need a new disk format, but not for this.

 

A separate launch file makes more sense, but will run into setting differences between emulators. For instance, emulators differ in whether they have separate XL and XE settings and what the difference is. IIRC, Atari800 has a single XL/XE setting, Atari++ enables extended memory in XE mode, and Altirra enables floating data bus in XE mode.

 

Very basic settings like BASIC enabled/disabled and 800/XL mode are doable, but at that point it would be more useful simply to check that in the program itself: auto-disable BASIC and report a startup error if the system or memory configuration isn't supported. This would help on real hardware as well as emulation, and Ice-T is a good example. Unfortunately, very few programs do this, and we don't have a good source for people to learn how to do these kinds of things, either.

 

Side note, I'd also highly discourage XML for any interchange format like this for complexity overkill reasons. Look up "internal DTD subset" for why.

  • Like 4

Share this post


Link to post
Share on other sites

No, not another file format.

(...)

Just store the metadata in a separate file, describing the required system parameters (PAL/NTSC, memory etc), which images should be loaded into which drives, other typical metadata stuff (release year, version, author(s), ...) and so on and ZIP everything up.

Exactly my idea. It's actually on my long ToDo file of things-I-wanna-add-to-Atari800.

 

A separate launch file makes more sense, but will run into setting differences between emulators.

An emulator that doesn't support a particular option, could simply ignore it (and notify the user). Also, the metadata file's syntax shouldn't be based on capabilities of any particular emulator. Edited by Kr0tki

Share this post


Link to post
Share on other sites

Please inspect post

http://atariage.com/forums/topic/162372-his-dark-majesty/page-5#entry2953919

and following ones for background information...

Well, I did not inspect that post, but here are my two cents: The problem is that ATR is pretty messy, though can represent a lot of disk formats. Unfortunately, it is unable to represent copy protected disks that use various tricks, like sector skew or duplicate sectors. ATX does, but ATX is even messier, and does not even cover anything beyond SD-FM coded disks. So yes, it would be appreciated to have a format that just works, and not the junk we have now.

 

Anyhow, even though I would support this by adding it to Atari++, it does currently makes only limited sense to do so. Who converts the existing files?

Share this post


Link to post
Share on other sites

From a collectors POV I'd hate another format but from a user POV the idea that a file could hold all the data to self modify its running state to be as perfectly compatible as possible intrigues me but the notion of having to re weed out the horrendous number of duplicates that to this day I'm still struggling with puts me right off.

 

Conventional dupe finding techniques would have zero effect and it would be a per item issue or hoping the format converter would explain what version of a file was used?

 

Either way it sounds a lot of work..

Share this post


Link to post
Share on other sites

Well, let's face it: if we were using actual floppy disks, the onus would be on the user to select the appropriate OS/RAM/BASIC state before running the software. The only shortcoming in the ATR format I recently noticed is that one must intuit the PERCOM data by analysing the geometry of the disk image: i.e. there's no track count, etc in the ATR header. But this is no great problem. As for internal BASIC: it should be a basic (no pun intended) requirement that software disable it automatically when required. Nevertheless, something as simple as a different file extender (ATN, for example) would be enough to tell the host system that internal BASIC should be disabled.

 

That said, a new format would be fun to play with if only to see what would be possible, but support for the old file formats would have to be preserved, since we could never hope to completely eradicate the ATR in our lifetimes. For better or worse, it's the proliferating standard. So, rather than streamlining things, parallel support for any new disk image format would tend to bloat existing implementations, so on balance I'd be happy to leave well enough alone - especially after having spent a lot of hours catering for the idiosyncrasies of the existing ATR layout.

Share this post


Link to post
Share on other sites

It seems concepts are pretty much confused here. A disk image format should be exactly that: a "disk image format" and not an "emulator configuration file format". Whether the user loads the disk with or without BASIC enabled is up to the user, in the very same way it was up to the user with the original machine. Who knows which plans the user has? Second-guessing does not look like a wise idea for me.

  • Like 3

Share this post


Link to post
Share on other sites

If I had to name requirements for a replacement for ATR, they'd be: simple format with pointers and fixed blocks for minimal parsing, aligned sectors for direct use on real hardware, able to support 256 and 512 byte sectors, no ambiguity over boot sector layout, support protected disk imaging, have a spec document, and have a publicly available disk imager that produces it. Anything less than that isn't going to go anywhere with ATR and ATX already in play. Even then, it'd have to be used for something that the existing formats aren't good for, such as faster direct access from an Atari-IDE adapter to images in a FAT32 partition, or imaging DD protected disks.

 

With regard to an auto-configure file, what I would suggest is having that file be the "image" handled by the emulator, and then loading that is what brings everything else along for the ride including configuration and associated disks/tapes/etc. The reason that works for me, though, is that Altirra has a separate "Boot Image" command similar to Atari800WinPLus's Autoboot Image command that is suited for such a master bootstrap operation.

 

For existing disks, it'd probably be better to keep them in the existing formats and just store the associated info in aggregate in a database file. I was thinking of starting something like this for Altirra anyway for auto-detection and flagging of disk images with known problems -- basically, a much more expanded version of the compatibility list I've been maintaining in the help file. Nobody is going to want entire collections of images reformatted to a new format that no existing emulators or PC-Atari connection tools can read.

Share this post


Link to post
Share on other sites

Would something along the lines of a CRC32 database work - for ROMs it would be easiest, probably 1 or 2 per title, ie raw and cracked (although of course multiple crack versions exist for many titles).

For ATR, just cover the data portion for the common images that have single games.

 

If just the "known problem" titles were covered, ie - need OS-A, 400/800 only, >64K required that would reduce the overall workload to construct the database significantly.

Share this post


Link to post
Share on other sites

Personally, I like the idea of a 2nd file which then specifies the required settings. I also think a larger file may be in order.

 

To address the concern of the user not being able to override the settings, any emulator that uses the new file format must also have an option to ignore the file's recommendations also.

Share this post


Link to post
Share on other sites

It seems concepts are pretty much confused here. A disk image format should be exactly that: a "disk image format" and not an "emulator configuration file format".

 

I find both aspects important. The resulting requirements do not exclude each other, but should be respected in common.

 

 

Whether the user loads the disk with or without BASIC enabled is up to the user, in the very same way it was up to the user with the original machine. Who knows which plans the user has? Second-guessing does not look like a wise idea for me.

 

While the user should be able to overrule environmental settings, I don't see your point.

I mean we are Atarians, and I always felt (and still feel) it was/is an advantage not to enter "LOAD"*",8,1" as a precondition to express the "users will"...

It is much more convenient and user friendly to have something self-contained which runs out-of-the box and don't send the user on a long journey through settings which may allow to execute an application. And please don't forget the documental/preservation effect. We are here (more or less) experts, knowing the abilities and have an idea which tweaks could help running a game or make it look better. But most of us have more than 25 years experience. Beginners (and maybe we too - when we starting to become real old) would be happy about some 'advice'...

 

Independent from the result of the poll, I already find this discussion very insightful and valuable - so please don't stop with comments!

Edited by Irgendwer

Share this post


Link to post
Share on other sites

Side note, I'd also highly discourage XML for any interchange format like this for complexity overkill reasons. Look up "internal DTD subset" for why.

 

In the linked post, I mentioned MP3-ID3 tags, which are quite simple and tied to the data, which I regard as an advantage...

BTW: If you prefer something without the XML overkill there is even a standard developed by an 'Atarian': JSON... ( http://a8.fandal.cz/search.php?search=Douglas+Crockford&butt_details_x=x )

Edited by Irgendwer

Share this post


Link to post
Share on other sites

You guys ever hear of something called file streams? That's what I use. I write metadata into a separate stream. Software that calls the filename directly calls the default stream by default (Disk Data). I read the metadata from the second stream as needed to provide runtime configuration information.

Share this post


Link to post
Share on other sites

In the linked post, I mentioned MP3-ID3 tags, which are quite simple and tied to the data, which I regard as an advantage...

Since you mention ID3 tags, keep in mind what problems they caused back then to software and hardware that was not aware of them - they played noise for a fraction of a second or even crashed. I had a hardware MP3 decoder chip (Micronas 3507 or something like that, can't remember) that not really liked them. I absolutely don't like the idea to add such a thing to an ATR, we already have enough problems detecting all of the broken ATRs and trying to handle them...

 

A lot more important than the file format is to decide what kind of information should be stored in your metadata file/block/whatever and how the semantics of that information should be.

 

First of all, metadata should be static. You most certainly wouldn't want to add information that could possible change over time or isn't valid for 100% of the cases (like, for example, specific emulator settings like needing the SIO patch to be disabled in emulator X).

 

Concentrating on real, existing hardware would be a good approach, I guess. For example you could specify the minimum system requirements in a metadata file like this:

config.system.type=Atari 800XL
But, what does this actually mean? Does it work on XEGS, or an 130XE? Or do you want to specify multiple configuration choices, like "the software can run either on an 800XL with Rambo 320k upgrade, but not on an 130XE or an 800XL with 320k newell"?

 

Most likely this information is easier to represent in 2-3 lines within a README.TXT than in a formal way in some metadata file/block.

 

Simple things like "requires OS/A" or "needs at least 64k RAM" are easy, but after you go beyond that things get _very_ tricky.

 

so long,

 

Hias

Share this post


Link to post
Share on other sites

It seems concepts are pretty much confused here. A disk image format should be exactly that: a "disk image format" and not an "emulator configuration file format". Whether the user loads the disk with or without BASIC enabled is up to the user, in the very same way it was up to the user with the original machine. Who knows which plans the user has? Second-guessing does not look like a wise idea for me.

Exactly.

 

Most likely this information is easier to represent in 2-3 lines within a README.TXT than in a formal way in some metadata file/block.

 

Simple things like "requires OS/A" or "needs at least 64k RAM" are easy, but after you go beyond that things get _very_ tricky.

Quite so. And if ATRs were real disks, they'd also have sticky labels attached with instructions like "boot with Option pressed", etc; such cues might easily be incorporated into the ATR's filename, even.

 

It is much more convenient and user friendly to have something self-contained which runs out-of-the box and don't send the user on a long journey through settings which may allow to execute an application. And please don't forget the documental/preservation effect. We are here (more or less) experts, knowing the abilities and have an idea which tweaks could help running a game or make it look better. But most of us have more than 25 years experience. Beginners (and maybe we too - when we starting to become real old) would be happy about some 'advice'...

Surely there's a line between ease of use and over-sanitisation (or idiot-proofing, if you like), and I wonder if having to boot a game with Option pressed isn't integral to the retro computing experience. It's hardly an expert-level requirement, figuring this out, but if that's too much for the user, why is he bothering to mess with old computers in the first place? I recently started fiddling with CP/M in the WinAPE CPC emulator, and I found it difficult to boot some disks, since the system is unfamiliar to me. Getting stuff to work involved experimenting with a new system, and none of my "expertise" seemed to be much use. Nevertheless, I eventually figured it out, and that was part of the fun. :)

Edited by flashjazzcat

Share this post


Link to post
Share on other sites

Quite so. And if ATRs were real disks, they'd also have sticky labels attached with instructions like "boot with Option pressed", etc; such cues might easily be incorporated into the ATR's filename, even.

 

I take the opposite angle on labeling files. I put a [bAS] on the end of the filename if the disk requires BASIC to run, and assume everything else is machine machine language.

Share this post


Link to post
Share on other sites

I think changing any longstanding file format because some people can't figure out how to use an application is ridiculous. Either make the application simpler / more user friendly, or write a simple tutorial of how to set it up, use it, and what settings to try changing when something isn't working. If Avery was so inclined, I suppose he could make a simple mode for Altirra or something. But I certainly wouldn't expect him to do something like that -- He's already over-extended himself in helping the community in my opinion. With complex/capable programs comes a learning curve, as FJC said, whether or not you're an IT person, a programmer, or rocket scientist. Sure, an emulator for an Atari 8-Bit (and I include them all) can often be more difficult than the simple machines that they emulate. But that's because they do much more than any single machine does.

 

  • Like 1

Share this post


Link to post
Share on other sites

I take the opposite angle on labeling files. I put a [bAS] on the end of the filename if the disk requires BASIC to run, and assume everything else is machine machine language.

 

Actually I do the same with files that require more memory, PAL video mode, OS-B, or SIO patch to be turned off, i.e.: [256K], [PAL], [OS-B], [sIO].

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...