Jump to content

Photo

Do we need a new standard for disk/cart images?

ATR FILE CAR IMAGE EMULATION FORMAT

33 replies to this topic

Poll: Do we need a new standard for disk/cart images? (45 member(s) have cast votes)

ATR/CAR images don't cover environmental needs. What about a new format?

  1. I would appreciate and even actively support it. (6 votes [13.33%] - View)

    Percentage of vote: 13.33%

  2. I would appreciate that. (7 votes [15.56%] - View)

    Percentage of vote: 15.56%

  3. I don't mind. (2 votes [4.44%] - View)

    Percentage of vote: 4.44%

  4. I doubt, that this is a good idea. (9 votes [20.00%] - View)

    Percentage of vote: 20.00%

  5. No! Not another file format! (21 votes [46.67%] - View)

    Percentage of vote: 46.67%

Vote Guests cannot vote

#1 Irgendwer OFFLINE  

Irgendwer

    Stargunner

  • 1,467 posts
  • Location:Germany

Posted Sun Mar 23, 2014 5:07 AM

Please inspect post

http://atariage.com/...-5#entry2953919

and following ones for background information...


Edited by Irgendwer, Sun Mar 23, 2014 5:40 AM.


#2 flashjazzcat ONLINE  

flashjazzcat

    Quadrunner

  • 14,501 posts
  • Location:United Kingdom

Posted Sun Mar 23, 2014 6:23 AM

Torn between "No! Not another file format" and "I would appreciate and even actively support it". If any new format had a 512 byte header, I'd sure appreciate it. That would greatly simplify disk image handling in things like Ultimate 1MB PBI, and even allow for performance improvements (since disk image sectors would align with physical sectors).

#3 Rybags OFFLINE  

Rybags

    Gridrunner

  • 16,103 posts
  • Location:Australia

Posted Sun Mar 23, 2014 6:48 AM

Yes - but needs to be downward compatible.

 

To have environmental parameters that an emulator can use - at the minimum might eliminate a whole bunch of people complaining about things not working.

 

But once images get out in the wild, it'd probably take years for everyone to get up to date.



#4 HiassofT ONLINE  

HiassofT

    Stargunner

  • 1,140 posts
  • Location:Salzburg, Austria

Posted Sun Mar 23, 2014 8:08 AM

No, not another file format.

What about existing hard and software? How do you handle multi-disk images? What about software requiring both a CAR and an ATR? Multiple, stacked carts?

If I understand you correctly, this is all about adding some metadata to help emulator users and (maybe?) library software to help structuring your software archive.

Just store the metadata in a separate file, describing the required system parameters (PAL/NTSC, memory etc), which images should be loaded into which drives, other typical metadata stuff (release year, version, author(s), ...) and so on and ZIP everything up.

Then convince emulator authors to support your metadata file format (optionally with direct support for opening the ZIP file) and you are done. No need for yet another disk or cartridge image format.

BTW: This is basically the same approach as the JAR (java archive) format - a ZIP with all the java classes and resources plus a manifest file.

so long,

Hias

#5 Rybags OFFLINE  

Rybags

    Gridrunner

  • 16,103 posts
  • Location:Australia

Posted Sun Mar 23, 2014 8:21 AM

Bundle or seperate descriptor file has it's merits but probably not real useful on real machines using SIO2SD and the like.  Although generally real hardware you're on a fixed configuration and such metadata isn't really vital anyway.

 

But for me, if it can be a seamless addition that doesn't break anything then why not?



#6 Xuel OFFLINE  

Xuel

    Dragonstomper

  • 747 posts
  • Location:US

Posted Sun Mar 23, 2014 10:35 AM

I see this as a way to wrap existing formats like XEX, ROM, ATR, CAS, etc. Maybe look at SAP and SID/PSID formats for inspiration? Another possibility is to make a separate metadata file. That way you still have the actual file in an established format.

Edit: Just realized I repeated what was already said. Anyhow I lean towards a separate file.

Edited by Xuel, Sun Mar 23, 2014 10:49 AM.


#7 JAC! OFFLINE  

JAC!

    Stargunner

  • 1,833 posts
  • Always looking for GFX and MSX for my demos
  • Location:Lebach, Germany

Posted Sun Mar 23, 2014 1:07 PM

I'm also against changing the existing container formats (ATR/CAR/XEX). They fully do what they were intended to do and are widely supported.
If we need additional information, it should be added as separate file. Just like Altirra already today automatically looks up the "example.atdbg" "example.lst", "example.lab" files if you start "example.xex".

This file (suggestion ".ati") should be in an extensible text format with well defined encoding (e.g properties/xml) and should be able to optionally include:

- release information (producer, title, year, authors, ...)

- documentation (in multiple languages)

- environment requirements (PAL/NTSC, ROMs, RAM sizes etc,  as "required" or "supported")

 

It also gives you the simple choice if you want to run the executable in the "current" environment (click on the ATR) or in the required envionment (click on the ATI).

This would be a compatible, optional extension that might also be valueable on the real machine.  In that sense, I vote for "Yes".

 

 

ATI-AtariInformation.png



#8 phaeron OFFLINE  

phaeron

    River Patroller

  • 2,741 posts
  • Location:Bay Area, CA, USA

Posted Sun Mar 23, 2014 2:47 PM

No to extending the existing formats. For one thing, they all have different levels of extensibility: ATR is a fixed size header format, CAR is a chunk-based format, and XEX has no extensibility at all since it is the raw DOS executable format. Second, from a layering standpoint this doesn't belong in the image formats. It would mean that the disk/tape/EXE parser would have to be able to reconfigure global system parameters above it. Ick. We do need a new disk format, but not for this.

 

A separate launch file makes more sense, but will run into setting differences between emulators. For instance, emulators differ in whether they have separate XL and XE settings and what the difference is. IIRC, Atari800 has a single XL/XE setting, Atari++ enables extended memory in XE mode, and Altirra enables floating data bus in XE mode.

 

Very basic settings like BASIC enabled/disabled and 800/XL mode are doable, but at that point it would be more useful simply to check that in the program itself: auto-disable BASIC and report a startup error if the system or memory configuration isn't supported. This would help on real hardware as well as emulation, and Ice-T is a good example. Unfortunately, very few programs do this, and we don't have a good source for people to learn how to do these kinds of things, either.

 

Side note, I'd also highly discourage XML for any interchange format like this for complexity overkill reasons. Look up "internal DTD subset" for why.



#9 Kr0tki OFFLINE  

Kr0tki

    Stargunner

  • 1,133 posts
  • Location:Warszawa, Poland

Posted Mon Mar 24, 2014 1:50 AM

No, not another file format.
(...)
Just store the metadata in a separate file, describing the required system parameters (PAL/NTSC, memory etc), which images should be loaded into which drives, other typical metadata stuff (release year, version, author(s), ...) and so on and ZIP everything up.

Exactly my idea. It's actually on my long ToDo file of things-I-wanna-add-to-Atari800.

A separate launch file makes more sense, but will run into setting differences between emulators.

An emulator that doesn't support a particular option, could simply ignore it (and notify the user). Also, the metadata file's syntax shouldn't be based on capabilities of any particular emulator.

Edited by Kr0tki, Mon Mar 24, 2014 1:50 AM.


#10 Larry OFFLINE  

Larry

    River Patroller

  • 4,104 posts
  • Location:U.S. -- Midwest

Posted Mon Mar 24, 2014 6:05 AM

Looks thus far like the "new standard" needs some "campaign promises!"  :-D

 

-Larry



#11 thorfdbg OFFLINE  

thorfdbg

    Dragonstomper

  • 800 posts

Posted Tue Mar 25, 2014 4:43 AM

Please inspect post

http://atariage.com/...-5#entry2953919

and following ones for background information...

Well, I did not inspect that post, but here are my two cents: The problem is that ATR is pretty messy, though can represent a lot of disk formats. Unfortunately, it is unable to represent copy protected disks that use various tricks, like sector skew or duplicate sectors. ATX does, but ATX is even messier, and does not even cover anything beyond SD-FM coded disks. So yes, it would be appreciated to have a format that just works, and not the junk we have now.

 

Anyhow, even though I would support this by adding it to Atari++, it does currently makes only limited sense to do so. Who converts the existing files?



#12 Mclaneinc ONLINE  

Mclaneinc

    Retro Madman

  • 6,577 posts
  • Location:Northolt, UK

Posted Tue Mar 25, 2014 5:06 AM

From a collectors POV I'd hate another format but from a user POV the idea that a file could hold all the data to self modify its running state to be as perfectly compatible as possible intrigues me but the notion of having to re weed out the horrendous number of duplicates that to this day I'm still struggling with puts me right off.

 

Conventional dupe finding techniques would have zero effect and it would be a per item issue or hoping the format converter would explain what version of a file was used?

 

Either way it sounds a lot of work..



#13 flashjazzcat ONLINE  

flashjazzcat

    Quadrunner

  • 14,501 posts
  • Location:United Kingdom

Posted Tue Mar 25, 2014 5:58 AM

Well, let's face it: if we were using actual floppy disks, the onus would be on the user to select the appropriate OS/RAM/BASIC state before running the software. The only shortcoming in the ATR format I recently noticed is that one must intuit the PERCOM data by analysing the geometry of the disk image: i.e. there's no track count, etc in the ATR header. But this is no great problem. As for internal BASIC: it should be a basic (no pun intended) requirement that software disable it automatically when required. Nevertheless, something as simple as a different file extender (ATN, for example) would be enough to tell the host system that internal BASIC should be disabled.

That said, a new format would be fun to play with if only to see what would be possible, but support for the old file formats would have to be preserved, since we could never hope to completely eradicate the ATR in our lifetimes. For better or worse, it's the proliferating standard. So, rather than streamlining things, parallel support for any new disk image format would tend to bloat existing implementations, so on balance I'd be happy to leave well enough alone - especially after having spent a lot of hours catering for the idiosyncrasies of the existing ATR layout.

#14 thorfdbg OFFLINE  

thorfdbg

    Dragonstomper

  • 800 posts

Posted Tue Mar 25, 2014 4:53 PM

It seems concepts are pretty much confused here. A disk image format should be exactly that: a "disk image format" and not an "emulator configuration file format". Whether the user loads the disk with or without BASIC enabled is up to the user, in the very same way it was up to the user with the original machine. Who knows which plans the user has? Second-guessing does not look like a wise idea for me.



#15 phaeron OFFLINE  

phaeron

    River Patroller

  • 2,741 posts
  • Location:Bay Area, CA, USA

Posted Tue Mar 25, 2014 10:51 PM

If I had to name requirements for a replacement for ATR, they'd be: simple format with pointers and fixed blocks for minimal parsing, aligned sectors for direct use on real hardware, able to support 256 and 512 byte sectors, no ambiguity over boot sector layout, support protected disk imaging, have a spec document, and have a publicly available disk imager that produces it. Anything less than that isn't going to go anywhere with ATR and ATX already in play. Even then, it'd have to be used for something that the existing formats aren't good for, such as faster direct access from an Atari-IDE adapter to images in a FAT32 partition, or imaging DD protected disks.

 

With regard to an auto-configure file, what I would suggest is having that file be the "image" handled by the emulator, and then loading that is what brings everything else along for the ride including configuration and associated disks/tapes/etc. The reason that works for me, though, is that Altirra has a separate "Boot Image" command similar to Atari800WinPLus's Autoboot Image command that is suited for such a master bootstrap operation.

 

For existing disks, it'd probably be better to keep them in the existing formats and just store the associated info in aggregate in a database file. I was thinking of starting something like this for Altirra anyway for auto-detection and flagging of disk images with known problems -- basically, a much more expanded version of the compatibility list I've been maintaining in the help file. Nobody is going to want entire collections of images reformatted to a new format that no existing emulators or PC-Atari connection tools can read.



#16 Rybags OFFLINE  

Rybags

    Gridrunner

  • 16,103 posts
  • Location:Australia

Posted Tue Mar 25, 2014 11:08 PM

Would something along the lines of a CRC32 database work - for ROMs it would be easiest, probably 1 or 2 per title, ie raw and cracked (although of course multiple crack versions exist for many titles).

For ATR, just cover the data portion for the common images that have single games.

 

If just the "known problem" titles were covered, ie - need OS-A, 400/800 only, >64K required that would reduce the overall workload to construct the database significantly.



#17 snicklin OFFLINE  

snicklin

    River Patroller

  • 2,197 posts
  • Location:Australia

Posted Wed Mar 26, 2014 12:22 AM

Personally, I like the idea of a 2nd file which then specifies the required settings. I also think a larger file may be in order.

 

To address the concern of the user not being able to override the settings, any emulator that uses the new file format must also have an option to ignore the file's recommendations also.



#18 Irgendwer OFFLINE  

Irgendwer

    Stargunner

  • Topic Starter
  • 1,467 posts
  • Location:Germany

Posted Wed Mar 26, 2014 4:08 PM

It seems concepts are pretty much confused here. A disk image format should be exactly that: a "disk image format" and not an "emulator configuration file format".

 

I find both aspects important. The resulting requirements do not exclude each other, but should be respected in common.

 

 

Whether the user loads the disk with or without BASIC enabled is up to the user, in the very same way it was up to the user with the original machine. Who knows which plans the user has? Second-guessing does not look like a wise idea for me.

 

While the user should be able to overrule environmental settings, I don't see your point.

I mean we are Atarians, and I always felt (and still feel) it was/is an advantage not to enter "LOAD"*",8,1" as a precondition to express the "users will"...

It is much more convenient and user friendly to have something self-contained which runs out-of-the box and don't send the user on a long journey through settings which may allow to execute an application. And please don't forget the documental/preservation effect. We are here (more or less) experts, knowing the abilities and have an idea which tweaks could help running a game or make it look better. But most of us have more than 25 years experience. Beginners (and maybe we too - when we starting to become real old) would be happy about some 'advice'...  

 

Independent from the result of the poll, I already find this discussion very insightful and valuable - so please don't stop with comments!


Edited by Irgendwer, Wed Mar 26, 2014 4:11 PM.


#19 Irgendwer OFFLINE  

Irgendwer

    Stargunner

  • Topic Starter
  • 1,467 posts
  • Location:Germany

Posted Wed Mar 26, 2014 4:22 PM

Side note, I'd also highly discourage XML for any interchange format like this for complexity overkill reasons. Look up "internal DTD subset" for why.

 

In the linked post, I mentioned MP3-ID3 tags, which are quite simple and tied to the data, which I regard as an advantage...

BTW: If you prefer something without the XML overkill there is even a standard developed by an 'Atarian': JSON... ( http://a8.fandal.cz/...t_details_x=x )


Edited by Irgendwer, Wed Mar 26, 2014 4:24 PM.


#20 pixelmischief OFFLINE  

pixelmischief

    Stargunner

  • 1,280 posts

Posted Wed Mar 26, 2014 5:32 PM

You guys ever hear of something called file streams?  That's what I use.  I write metadata into a separate stream.  Software that calls the filename directly calls the default stream by default (Disk Data).  I read the metadata from the second stream as needed to provide runtime configuration information.



#21 HiassofT ONLINE  

HiassofT

    Stargunner

  • 1,140 posts
  • Location:Salzburg, Austria

Posted Wed Mar 26, 2014 5:38 PM

In the linked post, I mentioned MP3-ID3 tags, which are quite simple and tied to the data, which I regard as an advantage...

Since you mention ID3 tags, keep in mind what problems they caused back then to software and hardware that was not aware of them - they played noise for a fraction of a second or even crashed. I had a hardware MP3 decoder chip (Micronas 3507 or something like that, can't remember) that not really liked them. I absolutely don't like the idea to add such a thing to an ATR, we already have enough problems detecting all of the broken ATRs and trying to handle them...

A lot more important than the file format is to decide what kind of information should be stored in your metadata file/block/whatever and how the semantics of that information should be.

First of all, metadata should be static. You most certainly wouldn't want to add information that could possible change over time or isn't valid for 100% of the cases (like, for example, specific emulator settings like needing the SIO patch to be disabled in emulator X).

Concentrating on real, existing hardware would be a good approach, I guess. For example you could specify the minimum system requirements in a metadata file like this:
config.system.type=Atari 800XL
But, what does this actually mean? Does it work on XEGS, or an 130XE? Or do you want to specify multiple configuration choices, like "the software can run either on an 800XL with Rambo 320k upgrade, but not on an 130XE or an 800XL with 320k newell"?

Most likely this information is easier to represent in 2-3 lines within a README.TXT than in a formal way in some metadata file/block.

Simple things like "requires OS/A" or "needs at least 64k RAM" are easy, but after you go beyond that things get _very_ tricky.

so long,

Hias

#22 flashjazzcat ONLINE  

flashjazzcat

    Quadrunner

  • 14,501 posts
  • Location:United Kingdom

Posted Wed Mar 26, 2014 6:43 PM

It seems concepts are pretty much confused here. A disk image format should be exactly that: a "disk image format" and not an "emulator configuration file format". Whether the user loads the disk with or without BASIC enabled is up to the user, in the very same way it was up to the user with the original machine. Who knows which plans the user has? Second-guessing does not look like a wise idea for me.


Exactly.
 

Most likely this information is easier to represent in 2-3 lines within a README.TXT than in a formal way in some metadata file/block.

Simple things like "requires OS/A" or "needs at least 64k RAM" are easy, but after you go beyond that things get _very_ tricky.


Quite so. And if ATRs were real disks, they'd also have sticky labels attached with instructions like "boot with Option pressed", etc; such cues might easily be incorporated into the ATR's filename, even.
 

It is much more convenient and user friendly to have something self-contained which runs out-of-the box and don't send the user on a long journey through settings which may allow to execute an application. And please don't forget the documental/preservation effect. We are here (more or less) experts, knowing the abilities and have an idea which tweaks could help running a game or make it look better. But most of us have more than 25 years experience. Beginners (and maybe we too - when we starting to become real old) would be happy about some 'advice'...


Surely there's a line between ease of use and over-sanitisation (or idiot-proofing, if you like), and I wonder if having to boot a game with Option pressed isn't integral to the retro computing experience. It's hardly an expert-level requirement, figuring this out, but if that's too much for the user, why is he bothering to mess with old computers in the first place? I recently started fiddling with CP/M in the WinAPE CPC emulator, and I found it difficult to boot some disks, since the system is unfamiliar to me. Getting stuff to work involved experimenting with a new system, and none of my "expertise" seemed to be much use. Nevertheless, I eventually figured it out, and that was part of the fun. :)

Edited by flashjazzcat, Wed Mar 26, 2014 6:45 PM.


#23 MrFish OFFLINE  

MrFish

  • 5,474 posts

Posted Wed Mar 26, 2014 7:57 PM

Quite so. And if ATRs were real disks, they'd also have sticky labels attached with instructions like "boot with Option pressed", etc; such cues might easily be incorporated into the ATR's filename, even.

 

I take the opposite angle on labeling files. I put a [BAS] on the end of the filename if the disk requires BASIC to run, and assume everything else is machine machine language.



#24 MrFish OFFLINE  

MrFish

  • 5,474 posts

Posted Wed Mar 26, 2014 8:16 PM

I think changing any longstanding file format because some people can't figure out how to use an application is ridiculous. Either make the application simpler / more user friendly, or write a simple tutorial of how to set it up, use it, and what settings to try changing when something isn't working. If Avery was so inclined, I suppose he could make a simple mode for Altirra or something. But I certainly wouldn't expect him to do something like that -- He's already over-extended himself in helping the community in my opinion. With complex/capable programs comes a learning curve, as FJC said, whether or not you're an IT person, a programmer, or rocket scientist. Sure, an emulator for an Atari 8-Bit (and I include them all) can often be more difficult than the simple machines that they emulate. But that's because they do much more than any single machine does.

 



#25 MrFish OFFLINE  

MrFish

  • 5,474 posts

Posted Wed Mar 26, 2014 9:29 PM

I take the opposite angle on labeling files. I put a [BAS] on the end of the filename if the disk requires BASIC to run, and assume everything else is machine machine language.

 

Actually I do the same with files that require more memory, PAL video mode, OS-B, or SIO patch to be turned off, i.e.: [256K], [PAL], [OS-B], [SIO].







Also tagged with one or more of these keywords: ATR, FILE, CAR, IMAGE, EMULATION, FORMAT

0 user(s) are browsing this forum

0 members, 0 guests, 0 anonymous users