Atari Info in Database Format

[neglectoru]

Resources like AtariAge are really great, but is such information available in "database" format somewhere? I really love the Gamebase model, where there is an actual database (an MDB file, in the Gamebase example) from which one can write one's own frontend.

 

In particular, I am searching for a good list of atari games. The three pieces of information I'm most interested in is

 

• Game Name
  
• Publisher
  
• Release Date
  

 

Any database format (even an Excel spredsheet or CSV file) would be greatly appreciated.

 

Do such databases of Atari 2600 games exist in any format?

[Hornpipe2]

Resources like AtariAge are really great, but is such information available in "database" format somewhere? I really love the Gamebase model, where there is an actual database (an MDB file, in the Gamebase example) from which one can write one's own frontend.

 

In particular, I am searching for a good list of atari games. The three pieces of information I'm most interested in is

 

• Game Name
  
• Publisher
  
• Release Date
  

 

Any database format (even an Excel spredsheet or CSV file) would be greatly appreciated.

 

Do such databases of Atari 2600 games exist in any format?

 

I haven't seen anything like that before, though I imagine it would be handy.

 

You might try your hand at writing a script to parse the stella.pro file ("properties" file for the Stella emulator):

http://stella.svn.sourceforge.net/viewvc/stella/trunk/src/emucore/stella.pro?revision=1870&view=markup

It has at least Cartridge MD5, name, manufacturer, and sometimes additional info (model #, controller types, sparse rarity info, AKA for clones / rereleases). The release year is usually in parentheses in the Cartridge name.

Edited February 21, 2010 by Hornpipe2

[Rom Hunter]

Are you looking for a PAL or NTSC list?

 

Also, are you looking for original releases only?

 

[neglectoru]

The Stella list is a good start, thanks! Any idea where it originated?

 

I am most interested in NTSC games. I know there are tons, and plenty of "unlicenced" games. A list like the one in the atariage site would be ideal. I bet this site is backed by a database...

[Rom Hunter]

The Stella list is a good start, thanks! Any idea where it originated?

Most of the info in the Stella.pro file is based on my collected ROM info:

http://www.atarimania.com/rom_collection_archive_atari_2600_roms.html

 

I am most interested in NTSC games. I know there are tons, and plenty of "unlicenced" games. A list like the one in the atariage site would be ideal. I bet this site is backed by a database...

Here are all known to exist original NTSC games (including prototypes):

http://www.atarimania.com/list_games_atari-2600-vcs-p_total-page-step-format-original_536-1-200-2-1_2_G.html

 

Edited February 23, 2010 by Rom Hunter

[Hornpipe2]

Rom - I believe he is looking for a downloadable database or spreadsheet with game information. Atarimania provides the same info but is web-based and can't be saved locally for doing custom SQL queries, enhancing emulator frontends with lots of information, etc.

 

neglectoru - Another option would be to download the entire Rom's Collection from Atarimania, list the filenames into a text file, and then try to use regexes on the filenames to produce a CSV of info. Most filenames include a rom name and manufacturer, and PAL releases have (PAL) appended... many others have a year (or even exact date) of release, controller info, programmer's names, AKA (for clones / pirates), prototype status, etc. Since version 4 (or is it 5?), Originals are marked with a ~ at the end. Link: http://www.atarimania.com/rom_collection_archive_atari_2600_roms.html

Edited February 23, 2010 by Hornpipe2

[Rom Hunter]

Since version 4 (or is it 5?), Originals are marked with a ~ at the end. Link: http://www.atarimania.com/rom_collection_archive_atari_2600_roms.html

V5, the one that's currently online.

 

V6 will appear later this year.

 

[Pioneer4x4]

Since version 4 (or is it 5?), Originals are marked with a ~ at the end. Link: http://www.atarimania.com/rom_collection_archive_atari_2600_roms.html

V5, the one that's currently online.

 

V6 will appear later this year.

 

Can the symbol be changed to something other than a tilde? I don't think you can do searches for ~ correctly. I used a file rename util to change them, then I could search on the new character.

[Rom Hunter]

I have no problems searching for the tilde.

 

When does this problem arise?

 

[+stephena]

As mentioned above, the stella.pro file is available from Stella SVN here. However, this isn't really in computer-readable format.

 

There's a PERL script in Stella that converts from stella.pro to a C-style array, located here.

 

The stella.pro file is automatically converted and compiled into Stella, which is also in SVN, located here.

 

So there are several options. You can work from the human-readable stella.pro, or convert the C-style array into a proper CSV (comma separated value) list that can be read directly into a spreadsheet or database. In fact, you could even modify the PERL script to do that for you.

 

If you'll be generating a CSV list, the headings are as follows:

"Cartridge.MD5"           => 0
"Cartridge.Manufacturer"  => 1
"Cartridge.ModelNo"       => 2
"Cartridge.Name"          => 3
"Cartridge.Note"          => 4
"Cartridge.Rarity"        => 5
"Cartridge.Sound"         => 6
"Cartridge.Type"          => 7
"Console.LeftDifficulty"  => 8
"Console.RightDifficulty" => 9
"Console.TelevisionType"  => 10
"Console.SwapPorts"       => 11
"Controller.Left"         => 12
"Controller.Right"        => 13
"Controller.SwapPaddles"  => 14
"Display.Format"          => 15
"Display.YStart"          => 16
"Display.Height"          => 17
"Display.Phosphor"        => 18
"Display.PPBlend"         => 19

 

Finally, there's also an undocumented commandline argument for Stella, '-listrominfo'. If launched with this argument, Stella will output each 'record' (ROM info) on a separate line, separated by the '|' character. Note that this currently doesn't output all the properties, only the most common stuff (md5, name, etc). Extending Stella to print out everything would be very easy, though. I can point you to the exact lines of code to modify

 

EDIT: I went ahead and made the changes in Stella, where the output from '-listrominfo' generates all properties for each ROM. Below is the resulting text file, and a database of that file in Access 2003 MDB format:

 

Text format from 'listrominfo'

Access 2003 MDB database

 

Note that these are on my personal space at work, so please don't post everywhere on the Internet.

[Hornpipe2]

Well there you go. Ask and ye shall receive : )

 

Stephen - will this become a permanent change? Some emulator frontends use Mame's -listxml command to dynamically set themselves up with game lists at first run... probably exactly what neglectoru is trying to do with a static database file. A fully explained and parsable -listrominfo would be of great benefit to frontend authors everywhere.

[+stephena]

Well there you go. Ask and ye shall receive : )

 

Stephen - will this become a permanent change? Some emulator frontends use Mame's -listxml command to dynamically set themselves up with game lists at first run... probably exactly what neglectoru is trying to do with a static database file. A fully explained and parsable -listrominfo would be of great benefit to frontend authors everywhere.

Yes, the 'listrominfo' command was inspired by how MAME does it. In the most recent release (3.0), it only lists some of the information. But I've already committed code that prints out all info about a ROM, and this will be included in the next release of Stella. As well, I just added code to print a 'header' line at the beginning of the list, so you know what each column corresponds to. This output can be read directly into Access to create an appropriate table. Also, since the fields are separated by the '|' character, it would be very easy for frontend authors to parse it (basically, just split on each '|').

[Rom Hunter]

Stephena,

 

Will any old era overdump and bad dump not in my collection still get info in Stella.pro?

 

If so, wouldn't it be better to delete this info completely?

 

This way ROM collectors will eventually get a clean and accurate collection of verified ROMs and they will notice in an instant if a dump is genuine (info) or not (no info).

 

I ask this, because there are many, many bugged and useless ROMs floating around.

 

If these will contain (outdated and incorrect) info, people will still put this stuff in their collection, thinking it might be important.

 

Edited February 24, 2010 by Rom Hunter

[+stephena]

Stephena,

 

Will any old era overdump and bad dump not in my collection still get info in Stella.pro?

 

If so, wouldn't it be better to delete this info completely?

 

This way ROM collectors will eventually get a clean and accurate collection of verified ROMs and they will notice in an instant if a dump is genuine (info) or not (no info).

 

I ask this, because there are many, many bugged and useless ROMs floating around.

 

If these will contain (outdated and incorrect) info, people will still put this stuff in their collection, thinking it might be important.

 

The internal database now includes somewhere around 3250 entries. Since your collection only includes ~1700 ROMs, the answer is yes, there's definitely some extra stuff in there. I remove bad dumps as soon as I find them, but overdumps might still be considered valid. And there's many homebrew and demos that, while not part of your collection, still deserve a mention. I believe you attempted to do a demo/homebrew list at some point, but gave up because of the number of ROMs out there. Stella.pro already includes that info, so it would be a waste to throw it away.

 

I'm not opposed to marking your ROMs with a certain character (just as you use '~'), but at this point in time I don't want to completely erase all the other stuff in the database. Some cleaning could be done, but not a complete removal. I don't think I'd be opposed to removing overdumps, though, since they're bad versions of ROMs that already exist. Demos/homebrews are unique, and should be preserved.

[Hornpipe2]

The internal database now includes somewhere around 3250 entries. Since your collection only includes ~1700 ROMs, the answer is yes, there's definitely some extra stuff in there. I remove bad dumps as soon as I find them, but overdumps might still be considered valid. And there's many homebrew and demos that, while not part of your collection, still deserve a mention. I believe you attempted to do a demo/homebrew list at some point, but gave up because of the number of ROMs out there. Stella.pro already includes that info, so it would be a waste to throw it away.

 

I'm not opposed to marking your ROMs with a certain character (just as you use '~'), but at this point in time I don't want to completely erase all the other stuff in the database. Some cleaning could be done, but not a complete removal. I don't think I'd be opposed to removing overdumps, though, since they're bad versions of ROMs that already exist. Demos/homebrews are unique, and should be preserved.

 

How closely tied are the UI/launcher and the emulator? I don't feel Stella itself should come with any sort of whitelist, nor should it even care about what is being played on it. The only reason it has to care would be knowing what kind of controllers or hardware are connected, and the bankswitch type to emulate... both of which could be supplied on command line. (One might contrast this approach to MAME (which does whitelist or at least track rom info), but remember that MAME is a giant duct-taped together collection of emulators and choosing a romset actually implies choosing a driver as well)

 

Would it be a bad idea to make Stella into a command-line application which is called from a frontend, and in that frontend is where stella.pro (or in the distant future, stella.xml) lives? On GUI-based systems you could put them together into the single app.

 

In my ideal world 2600 .bins would have a 128-byte header at the front which described bankswitch type, controller inputs, name, etc. thus rendering this entire discussion moot. : )

[+stephena]

The internal database now includes somewhere around 3250 entries. Since your collection only includes ~1700 ROMs, the answer is yes, there's definitely some extra stuff in there. I remove bad dumps as soon as I find them, but overdumps might still be considered valid. And there's many homebrew and demos that, while not part of your collection, still deserve a mention. I believe you attempted to do a demo/homebrew list at some point, but gave up because of the number of ROMs out there. Stella.pro already includes that info, so it would be a waste to throw it away.

 

I'm not opposed to marking your ROMs with a certain character (just as you use '~'), but at this point in time I don't want to completely erase all the other stuff in the database. Some cleaning could be done, but not a complete removal. I don't think I'd be opposed to removing overdumps, though, since they're bad versions of ROMs that already exist. Demos/homebrews are unique, and should be preserved.

 

How closely tied are the UI/launcher and the emulator? I don't feel Stella itself should come with any sort of whitelist, nor should it even care about what is being played on it. The only reason it has to care would be knowing what kind of controllers or hardware are connected, and the bankswitch type to emulate... both of which could be supplied on command line. (One might contrast this approach to MAME (which does whitelist or at least track rom info), but remember that MAME is a giant duct-taped together collection of emulators and choosing a romset actually implies choosing a driver as well)

 

Would it be a bad idea to make Stella into a command-line application which is called from a frontend, and in that frontend is where stella.pro (or in the distant future, stella.xml) lives? On GUI-based systems you could put them together into the single app.

Stella started out as a commandline app, with the ROM launcher added later. There's a commandline argument 'uselauncher', which can be set to false. In this case, it starts and exits as if it had no ROM launcher at all. I believe this is exactly what you're looking for. And every single option that can be set in the properties file can also be set from the commandline; the manual is your friend in this case. I should also add that the database isn't a whitelist; it just contains extra info for those ROMs that need it. Stella will run anything you throw at it, trying to autodetect as much as possible.

 

In my ideal world 2600 .bins would have a 128-byte header at the front which described bankswitch type, controller inputs, name, etc. thus rendering this entire discussion moot. : )

This was talked about over the years, but nothing ever came of it. At this point in time, I'm not sure what it would accomplish. If added, it would break ROMs for every other emulator. And even if that were acceptable, Stella would still need facilities to autodetect and/or override everything in the header. As well, the autodetection in Stella has become quite advanced over the past year or so. In over 4000 tested ROMs, the bankswitch autodetection only failed for 7 of them (and several of those were bad dumps). Similar success rates occur for NTSC/PAL autodetection, and both types of autodetection are constantly being improved. So I'd say that with all the drawbacks of adding a header, and the relatively few advantages, there's probably not much point.

[Rom Hunter]

The internal database now includes somewhere around 3250 entries. Since your collection only includes ~1700 ROMs, the answer is yes, there's definitely some extra stuff in there. I remove bad dumps as soon as I find them, but overdumps might still be considered valid. And there's many homebrew and demos that, while not part of your collection, still deserve a mention. I believe you attempted to do a demo/homebrew list at some point, but gave up because of the number of ROMs out there. Stella.pro already includes that info, so it would be a waste to throw it away.

 

I'm not opposed to marking your ROMs with a certain character (just as you use '~'), but at this point in time I don't want to completely erase all the other stuff in the database. Some cleaning could be done, but not a complete removal. I don't think I'd be opposed to removing overdumps, though, since they're bad versions of ROMs that already exist. Demos/homebrews are unique, and should be preserved.

Exactly my thoughts.

 

BTW: I wasn't talking about demo's, hacks and homebrews. Of course they should have their own ROM info. I was talking about all the useless bad dumps and overdumps that are still infecting clean ROM sets.

 

[neglectoru]

 

EDIT: I went ahead and made the changes in Stella, where the output from '-listrominfo' generates all properties for each ROM. Below is the resulting text file, and a database of that file in Access 2003 MDB format:

 

Text format from 'listrominfo'

Access 2003 MDB database

 

Note that these are on my personal space at work, so please don't post everywhere on the Internet.

 

Wow, perfect. Many thanks!

[+stephena]

Stephena,

 

Will any old era overdump and bad dump not in my collection still get info in Stella.pro?

 

If so, wouldn't it be better to delete this info completely?

 

This way ROM collectors will eventually get a clean and accurate collection of verified ROMs and they will notice in an instant if a dump is genuine (info) or not (no info).

 

I ask this, because there are many, many bugged and useless ROMs floating around.

 

If these will contain (outdated and incorrect) info, people will still put this stuff in their collection, thinking it might be important.

 

After further thought, and looking at the ROMs from the most recent hacks & homebrews thread, I may just go with your suggestion and clear the entire database of stuff that isn't in your official release. I thought the extra info was valid, but as I'm seeing now, much of it is incorrect, and it's missing at least 600 ROMs anyway. And it's enough of a burden to maintain it with just the ROMs you post, not even counting the 1000's of others out there.

 

So, perhaps in the next release or two I'll move to RomHunter-approved ROMs only. Note that this isn't a whitelist. Stella will still run with whatever you throw at it, it just won't contain all the info internally. If someone wants to take on the task of categorizing 3000+ ROMs into an up-to-date database, of course I'll consider adding it to Stella again. But my guess is that (unlike myself), nobody is crazy enough to do so

 

Besides, this extra maintenance takes away from Stella development time. I think most people would agree that as the only developer working on Stella right now, my time is better spent on the emulator itself.

[Omegamatrix]

I agree 100% of your time should be working on Stella instead of maintaining the stella.pro files.

 

 

On that topic of titles, why does Stella list the year? I don't mind seeing the name of the game and the manufacture, but the year seems unnecessary IMO.

[Rom Hunter]

If someone wants to take on the task of categorizing 3000+ ROMs into an up-to-date database, of course I'll consider adding it to Stella again. But my guess is that (unlike myself), nobody is crazy enough to do so

My guess as well.

 

Please know that my ROM collection only contains ROMs that were released between 1977 and 1992.

 

Edited March 2, 2010 by Rom Hunter