MemCard file area: file types proposal

[brpocock]

For "CiE," we'll likely be allowing multiple user "files" to be stored on one MemCard.

 

Apparently the file area has never been used before, so I have a proposal to structure the blocks somewhat to prevent collisions between programs.

 

Creating a directory structure and the like may be overkill, but here's the general idea I'm working from:

 

Each pair of 64B blocks is "ganged" into a single 128B "file."

 

Each "file" block begins with 8 bytes of "file type" data, followed by 8 bytes of "file name" data, in 7-bit ASCII, padded with blanks (hex $20). Valid filename characters are ASCII A-Z, 0-9, "-" (minus/hyphen), and $20/space. Other characters may or may not be supported.

 

Bytes 0 and 8 (the first and ninth byte, the first filetype and first filename byte) are both $ff if the block is available.

 

Each program that supports user-named files should have basically the same actions:

 

1. To CREATE a file, scan from the start of the file area until finding a block where bytes 0 & 8 are both $ff, then write out an 8-byte, fixed, filetype code, and the 8-byte filename.

 

2. To WRITE a file, look for the 16 bytes file type + file name and then write over it. Files taking more than 112 data bytes should write the same 16B header on each subsequent file block they use, so that other "dumb" software needn't know about it.

 

3. To DELETE a file, same. Blanking out (to $ff) bytes 0 through 8 is preferred, i.e. writing 9 "blank" bytes. If deleting a multiblock file, every block with the same file type and file name should be deleted.

 

4. To READ a file, same game, but keep reading for 128B.

 

5. To SEARCH for a file, scan the first 16B of every 128B file block.

 

The 8-byte file type should be treated as an 8×8 bitmap in "player" format. Programs accessing the MemCard may display the bitmap for each file type.

 

For example: CiE's main menu might have a DELETE FILE (or, in NES terms, ELIMINATION MODE) entry. CiE doesn't need to know about the file structure of various other games, but it can display the icon and file name on the deletion menu or list.

 

This provides some verification of file types, so CiE won't accidentally try to load data saved by another app. It also allows files with the same names, but different types. Given that most Atari games are likely to only use initials, or short (4, 6, 8 byte) player names, odds are good that one person will use the same filename for every game. Having a dozen files named "BRPOCOCK" on one MemCard won't be a problem in this situation.

 

Sticking to a limited character range will make it more likely that filenames can display in various games. Perhaps games with restricted fonts (i.e. not full ASCII 7bit font ranges, i.e. most games) should display any character not present in their font as some "identifiable" character meaning "other character I can't draw" -- e.g. a hollow box should be fairly doable, even in a 3px × 5px font...

 

@@@
@.@
@.@
@.@
@@@

 

So, my questions:

 

Any comments, suggestions, improvements on the above?

 

European users in particular: is being stuck with A-Z too unbearable? Are there a very few additional characters that are absolutely necessary? I don't speak the languages, but I gather ø and ß might be necessary?

 

Is it necessary to distinguish between I and 1, O and 0, Z and 2, and S and 5, or should we just store those using their numeric forms since most 2600 games aren't going to distinguish between them on screen?

 

Is anyone else even planning to use the named file area or should I just run wild?

 

EDIT: $ff for blank blocks, not $00 as I'd initially written; and fixed typo, 112B of each 128B would be free after 16B header.

Edited February 1, 2006 by brpocock

[+batari]

Each "file" block begins with 8 bytes of "file type" data, followed by 8 bytes of "file name" data, in 7-bit ASCII, padded with blanks (hex $20). Valid filename characters are ASCII A-Z, 0-9, "-" (minus/hyphen), and $20/space. Other characters may or may not be supported.

I'm sure it's a good idea. Actually, this is what was stopping me from finishing up bB support for the memcard, as I feared that I'd need to figure this out myself. That said, 8 bytes for the file type seems to be overkill. I can't see it needing more than 2. This allows for 65535 different types. Even with 2 bytes, the chances are sufficiently low that another person will not accidentally use your type. If you want to use 8 bytes of ASCII anyway, a hash function could compress them down to two.

 

The reason I want to save space here is due to my plans for supporting the memcard in bB. I am planning on implementing a "save game state" feature where a simple command will save or restore the entire contents of RAM, even in the middle of a game. It should work like saving your game in an emulator. I don't need to save the entire 128 bytes of RAM - I can probably make do with 118, but I can't deal with 112. I don't want to have to encroach on an additional block just for a few bytes.

[brpocock]

The reason I want to save space here is due to my plans for supporting the memcard in bB.  I am planning on implementing a "save game state" feature where a simple command will save or restore the entire contents of RAM, even in the middle of a game.  It should work like saving your game in an emulator.  I don't need to save the entire 128 bytes of RAM - I can probably make do with 118, but I can't deal with 112.  I don't want to have to encroach on an additional block just for a few bytes.

1010951[/snapback]

 

8B was intended to allow a user-visible icon, but certainly is overkill. Realistically, one byte is probably all that's absolutely needed; I don't forsee 255 (plus $ff=deleted) different games needing file space.

 

...wouldn't the tempvars for MemCard access require 16B of temp RAM at least (including at least 8B for filename and 8B for the various other tempvars)?

[supercat]

Apparently the file area has never been used before, so I have a proposal to structure the blocks somewhat to prevent collisions between programs.

 

The one really big requirement for this area is to avoid collisions. Other than that, I don't really see much need to require that applications use filenames or anything else--that should be entirely up to the application.

 

Therefore, I would propose the following if nobody's yet done anything that would conflict with it:

 

-1- Every application that uses the "file storage area" must have at least one statically allocated page of EEPROM associated with it.

 

-2- The "ID" of an application is the address of its static page, divided by 256

 

-3- The first 6 bytes of such an application's first static page should contain its name using a reduced-character set (e.g. " 123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ.-"); the top bit of each byte should be masked off for display purposes.

 

-4- The file area is divided up into 128-byte records. The first byte of each record identifies its type:

 

FF -- Blank

04-BF -- Data owned by application with the specified ID (remaining 127 bytes may be used freely by the application)

00 -- Supplemental application info

 

-5- An application info record (optional) contains the following data:

 

Type (1 byte) holding the value 0

Application ID (1 byte) holding the application ID

Small icon (7 bytes, MSB at left) If 7 pixels high, leave bottom blank

Large icon (49 bytes (six 7-byte columns of data, followed by color)

Reserved (6 bytes, write as FF)

Reserved (64 bytes, write as FF)

 

It is suggested that applications should use some reasonable scheme to identify for their own purposes multiple files of data, but applications are not expected to do anything in particular with other applications' data.

 

I would officially earmark the memory area from $3000-$3FFF as application scratchpad (note that Strat-O-Gems already uses $3Fxx). Applications would be free to write whatever they want in that area, but should not expect such information to persist when the unit is powered down. A game like CIE could use that area to keep track of the current game; if the user selects "save", appropriate information would be copied into the "file area".

[vdub_bobby]

You know, a proposed file system has been hashed out on the [stella] list already. Not that it is the best system or anything (as you mention, no one has used the file area yet), but it might be nice to start the discussion from there, rather than beginning all over again (again).

 

Also, it might be nice to take this discussion to the [stella] list as well, since a lot of 2600 coders are more likely to see the discussion (and take) part there.

 

EDIT: Notably, IIRC, a lot of things like appropriate file name length, file identifiers, etc. were hashed out there with a lot of smart folks, like Richard H, Alex H, Thomas J, etc. and it seems like a waste to ignore that discussion and/or duplicate it.

Edited February 1, 2006 by vdub_bobby

[supercat]

Even with 2 bytes, the chances are sufficiently low that another person will not accidentally use your type.

 

For games that need a fairly small amount of storage, it would be simpler to ask for a static allocation. I'll admit I was greedy with Strat-O-Gems (hogging 4 pages) and in future games that need more than 1-2 pages should probably go into the file area, but I didn't see any real consensus about how the file area should work.

 

I'd suggest having Richard H. assign unique IDs; for any game which uses static allocation the static ID could be the address of that allocation.

 

The reason I want to save space here is due to my plans for supporting the memcard in bB.  I am planning on implementing a "save game state" feature where a simple command will save or restore the entire contents of RAM, even in the middle of a game.  It should work like saving your game in an emulator.  I don't need to save the entire 128 bytes of RAM - I can probably make do with 118, but I can't deal with 112.  I don't want to have to encroach on an additional block just for a few bytes.

1010951[/snapback]

 

IMHO, the right approach would be to require that the first byte of each block be the application ID; additional bytes would be up to the application. If the application data was such that it didn't need any extra information (e.g. you pull it up and allow the user to use the joystick to select "next", "prev", or "ok") then you don't need to waste space with filenames or anything like that.

 

If a limit of about 190 applications using the EEPROM would be objectionable, the format could be extended later to allow for two-byte application IDs without breaking any existing applications.

[brpocock]

...

1010954[/snapback]

 

Only downside I might see is that it makes it rougher to identify specific savegame data from another game if the area were to get full.

 

Ideally, games shouldn't be deleting one another's data, but realistically, as with e.g. the GCN, we have a disparity between game ROM and secondary storage. It's quite plausible to have e.g. a borrowed game's data "stuck" on a MemCard "forever" if there's no way to delete "foreign" data.

 

Since CiE is such a huge beast and I've got a bit of space to dedicate to a file manager, having the ability to delete arbitrary files would be nice. (Carefully buried on the OPTIONS menu with a default to "NO" on the confirmation, of course.) With no "filename" identifiers, the only info I can show will be the icon and app name.

 

That should suffice for most purposes, though, since if the player has the cart that created the file, s/he can use that card to browse the files and delete them singly using some more meaningful ID: player name, or whatever.

 

EG: Browsing CiE files from the CiE cart will show the player's filename or user name entered when they saved. If s/he finds an arbitrary file of type "METROD" and doesn't have access to a "METROD" tape to discover something more useful about that particular file, then deleting it "sight unseen" isn't really going to hurt anyone.

 

I'll hold off until the week-end at least before coding up anything, in case there are further refinements on the above, but I do like the plan. I'll see about creating some standard routines for accessing files using that structure, post them here, and submit them upstream to the AVox SDK.

[brpocock]

You know, a proposed file system has been hashed out on the [stella] list already.  Not that it is the best system or anything (as you mention, no one has used the file area yet), but it might be nice to start the discussion from there, rather than beginning all over again (again).

 

Also, it might be nice to take this discussion to the [stella] list as well, since a lot of 2600 coders are more likely to see the discussion (and take) part there.

 

EDIT: Notably, IIRC, a lot of things like appropriate file name length, file identifiers, etc. were hashed out there with a lot of smart folks, like Richard H, Alex H, Thomas J, etc. and it seems like a waste to ignore that discussion and/or duplicate it.

1010958[/snapback]

 

I'll try to dig up those archives tomorrow. I didn't notice such postings on a quick Googling, just the "first 8 bytes as filename" suggestion in the SDK docs, which leaves the file type collision issue wide open. Do you know by any chance approximately when the discussions were made?

[supercat]

You know, a proposed file system has been hashed out on the [stella] list already.  Not that it is the best system or anything (as you mention, no one has used the file area yet), but it might be nice to start the discussion from there, rather than beginning all over again (again).

1010958[/snapback]

 

I've seen that proposal. I frankly that requiring any sort of filename is massive overkill. IMHO, the only thing the standard needs to do is allow an application to classify blocks into three categories:

 

-1- Blank

-2- Mine

-3- Someone else's

 

That's it. Requiring anything beyond that at the 'standards' level is only going to discourage people from using the file area at all.

 

Consider a game like Strat-O-Gems. Were it not for my desire to get the thing shipped (and the fact that debugging's much harder when I can't use an emulator) I would have liked to include the ability to save an arbitrary number of "great plays" (instead of just the three record-setting ones). Had I done so, which would seem a more logical method of saving and selecting plays:

 

-1A- When user wants to save the last "great play", user pushes select. User then uses the joystick to enter a filename.

 

-1B- When user wants to save the last "great play", user punches select. Screen flashes "SAVING..." for about 1/10 second and then play continues.

 

-2A- To review a great play, user selects from a list of filenames.

 

-2B- Great plays show in "attract" mode; if user pushes left or right on the joystick, the next or previous great play will be shown.

 

It seems to me that 1B and 2B make a lot more sense than 1A and 2A.

 

For something like "Synthcart", filenames might make more sense. But even if the storage standard doesn't require filenames, applications would be free to implement them as they see fit.

 

The only feature I can see as having some value beyond the block classification ("black/mine/not-mine") would be a means by which applications could let a user clean out data from applications that were no longer used and not otherwise available. Even there, I don't see much point in requiring that applications do much to support such features; including a six-byte application name in the static area seems a fair approach, even if not a great one.

[supercat]

Only downside I might see is that it makes it rougher to identify specific savegame data from another game if the area were to get full.

1010970[/snapback]

 

That's true, but in many cases I don't think a filename would do much good. Suppose you see a bunch of Strat-O-Gems files called GAME0001, GAME0002, etc. would that tell you ANYTHING?

 

Since I've specified that the MSB's of the application name in the static area should be ignored when displaying them, that provides 6 bits to extend things; in addition, the extended application record provides lots of reserved space to extend things. I would expect that things could be extended quite nicely without breaking any existing applications.

 

One bit, for example, would indicate whether the application used any "files" bigger than one block. Another would indicate whether the application's files should start with a filename. For multi-block files, I would suggest the following standard:

 

Creator ID (1 byte)

File ID (1 bytes)

Sequence number (1 byte)

Next-block pointer (1 byte)

Filename (if using filenames, and sequence number is zero); name should be designed so first six characters are meaningful, but name may be up to 59 characters (scrolled). Last character should have MSB set.

 

This would allow PC-based applications (using the plug-in interface) to load or save files intelligently.

[+batari]

The reason I want to save space here is due to my plans for supporting the memcard in bB.  I am planning on implementing a "save game state" feature where a simple command will save or restore the entire contents of RAM, even in the middle of a game.  It should work like saving your game in an emulator.  I don't need to save the entire 128 bytes of RAM - I can probably make do with 118, but I can't deal with 112.  I don't want to have to encroach on an additional block just for a few bytes.

1010951[/snapback]

 

8B was intended to allow a user-visible icon, but certainly is overkill. Realistically, one byte is probably all that's absolutely needed; I don't forsee 255 (plus $ff=deleted) different games needing file space.

 

...wouldn't the tempvars for MemCard access require 16B of temp RAM at least (including at least 8B for filename and 8B for the various other tempvars)?

1010952[/snapback]

No, the filename can be compiled into ROM. I won't be putting up a file manager menu or anything - just a simple mechanism to read and write.

 

You know, a proposed file system has been hashed out on the [stella] list already.  Not that it is the best system or anything (as you mention, no one has used the file area yet), but it might be nice to start the discussion from there, rather than beginning all over again (again).

I've seen that proposal. I frankly that requiring any sort of filename is massive overkill. IMHO, the only thing the standard needs to do is allow an application to classify blocks into three categories:

 

-1- Blank

-2- Mine

-3- Someone else's

 

That's it. Requiring anything beyond that at the 'standards' level is only going to discourage people from using the file area at all.

I agree with this. I'm not going to do anything complicated. If the standards are too difficuly/strict, I just won't follow them...

 

Now to find that discussion that I somehow missed.

 

Though I disagree with the static allocation thing. I don't want to require users to "apply" for static space on the memcard before they use it in their programs. I'd rather just have a "plug and play" driver that looked for free space and used it, and could read/write/erase that space depending on some identifier that the user has chosen for their program. Like a short filename... even just a few bytes would probably be enough.

[supercat]

Though I disagree with the static allocation thing.  I don't want to require users to "apply" for static space on the memcard before they use it in their programs.  I'd rather just have a "plug and play" driver that looked for free space and used it, and could read/write/erase that space depending on some identifier that the user has chosen for their program.  Like a short filename... even just a few bytes would probably be enough.

1010987[/snapback]

 

If allocations are registered, an 8-bit field will probably suffice; if not, there are ways of extending it. If allocations are not registered, even 4 bytes may not be enough.

 

Perhaps the "address" space of IDs could be allocated as follows:

 

08-BF -- Single-byte IDs corresponding to memory pages [the first 8 memory pages are already allocated to games that don't use file system space]

 

C000-CFFF -- Twelve-bit IDs, manually allocated

 

01xxxxxxxx -- 32-bit IDs (could be allocated by a web server; even if someone spammed the server it would be harmless)

 

If a system like this is used, the "program information" block should probably be designed to accommodate such ids (since there's no static block to hold even a six-character application name).

[+batari]

I read the discussion on [stella] and the original suggestion by Paul Slocum was perfectly reasonable (one byte type followed by 8 byte filename.) I plan to use the file area and not any sort of static allocation, so for my purposes, this is perfectly adequate.

 

However, I should note that the problem does not appear to have been resolved. So further discussion is defnintely warranted. So we should hammer this out here. Also I think here is appropriate because I found much of the rest of the [stella] discussion to be absurd. They were actually suggesting, among other things, Huffman encoding of filenames! No offense to the original authors, but that's just not going to happen. It's no surprise that nobody has used the file area. I can't imagine that avoiding collisions is that important.

 

Where is the "file area" exactly, and how big is it?

[supercat]

Also I think here is appropriate because I found much of the rest of the [stella] discussion to be absurd.... It's no surprise that nobody has used the file area.  I can't imagine that avoiding collisions is that important.

 

If avoiding collisions can be done easily (and it can) it should be done. Otherwise, if someone tries to use a memcard with one game that has previously been used by a certain other game, the memcard could become useless to both games.

 

The probability of collision may not be huge, but the potential annoyance is pretty large, so taking preventive measures seems entirely sensible.

 

Where is the "file area" exactly, and how big is it?

1011118[/snapback]

 

The memory consists of 512 64-byte pages. It's possible to write any portion of a page in a single write, but writes may not span pages. Memory may be read sequentially without regard for page boundaries.

 

Richard H's stated plan is to have the first 256 pages (16K) reserved for static allocation (e.g. Strat-O-Gems is permanently allocated pages 4-7 at addresses $0100-$01FF) and have the second 16K used as the file area. My suggested amendment to that proposal would be to reserve the last 64 pages of the first half (4K) as application scratchpad. This would reduce to 192 the number of pages available for static allocation, but I doubt that even that would get filled up.

 

Having a 4K scratchpad area would make it possible to implement a game like "Boggle" without any expansion RAM. If the dictionary size is 16,000 words, the game could use 2K of EEPROM to keep track of which words the player has found (one bit per dictionary word), and 2K to keep track of which words the computer has found. Storing things in such fashion would eliminate the need to search for and eliminate duplicate entries (which could otherwise be difficult and time consuming if words were simply stored as text).

 

A 4K area could also be very useful for an RPG, as it could hold the current "world" the player is walking around in. Small parts of the world would be read into RAM as needed for display, but the rest would be in EEPROM. Because the area would always be available and would represent sequential memory, random access would be practical.

[+batari]

If avoiding collisions can be done easily (and it can) it should be done. Otherwise, if someone tries to use a memcard with one game that has previously been used by a certain other game, the memcard could become useless to both games.

Really? How could it make the memcard unusable?

 

At least with the way I plan to use it in bB, this could not happen. One bB game cannot make another unable to use the memcard. Loading savegame data from a different bB game might cause the game to crash or do weird things, but it would not make the memcard unusable.

 

All I am going to do is have the user specify a filename to read/write to. Of course, in the docs I'd recommend using unique filenames (and not something like "FILE0001" or "SAVEGAME") and allow any byte (0-$FF) to be a part of the filename and not just ASCII.

 

If you really want to prevent collisions with bB games, I could just hard-code $BB in as the file type, so others could just avoid any file with $BB as the first byte.

[supercat]

If avoiding collisions can be done easily (and it can) it should be done. Otherwise, if someone tries to use a memcard with one game that has previously been used by a certain other game, the memcard could become useless to both games.

Really? How could it make the memcard unusable?

 

Suppose that Strat-O-Gems used the file area as I'd planned, and some other game happened to create a file that appeared before mine in EEPROM but shared my ID. When a user opts to load a file, Strat-O-Gems would read the other application's file and try to process it. One of the pieces of data in the file is the Y position at which the falling gem hit bottom. If a bogus number were to appear there and Strat-O-Gems didn't validate it, it could cause Strat-O-Gems to wipe out the wrong part of memory and crash.

 

While such dangers could be avoided using a "filename"-based system (there the crash would only occur if the user selected the wrong file to load), requiring that people use filenames rather than choosing files based upon what they actually contain would impose needless inconvenience on programmers and users alike.

 

At least with the way I plan to use it in bB, this could not happen.  One bB game cannot make another unable to use the memcard.  Loading savegame data from a different bB game might cause the game to crash or do weird things, but it would not make the memcard unusable.

 

All I am going to do is have the user specify a filename to read/write to.  Of course, in the docs I'd recommend using unique filenames (and not something like "FILE0001" or "SAVEGAME") and allow any byte (0-$FF) to be a part of the filename and not just ASCII.

 

How are users going to select a file to load? Is a user just going to randomly pick files until he finds one he likes? Unless filenames are user-generated, I don't see them as containing anything useful beyond the identity of the application that generated them, and if programmers register their IDs that could be stored just as well in a byte or two.

 

If you really want to prevent collisions with bB games, I could just hard-code $BB in as the file type, so others could just avoid any file with $BB as the first byte.

1011153[/snapback]

 

I'm not responsible for allocating such things, but if $BB wasn't allocated to anything other than bB games, and if bB provided its own means to arbitrate among different bB games, that would be fine. If a scheme like mine were adopted, bB games should write the string "BBASIC" [using an encoding such as I listed] at address $2EC0 so any application that wanted to show who owned what memory could do so.

[+batari]

While such dangers could be avoided using a "filename"-based system (there the crash would only occur if the user selected the wrong file to load), requiring that people use filenames rather than choosing files based upon what they actually contain would impose needless inconvenience on programmers and users alike.

 

...

 

How are users going to select a file to load?  Is a user just going to randomly pick files until he finds one he likes?  Unless filenames are user-generated, I don't see them as containing anything useful beyond the identity of the application that generated them, and if programmers register their IDs that could be stored just as well in a byte or two.

All I was going to do is implement a few commands. I'm keeping things simple:

 

save [filename]

load [filename]

erase [filename]

 

Or something of that sort. The bB programmer specifies a filename, up to 8 chars. save [filename] will first see if the file exists, if so it's overwritten. if not, it looks for blank space and puts the file in the first blank slot found (of course, it will only use the upper 16k.)

 

Each file just contains the entire RAM contents (except some space for temporary storage while loading/saving.)

 

A bB game will typically load or save just one file (the game state.)

 

If programmers want to add the ability to load or save multiple files, they (not me) need to implement some way to do this. There will be no "file manager" or anything. I will suggest a way to use multiple files in the docs, however. Like a screen where the user selects among several "savegame slots" but this ability will not be automatic.

 

If you really want to prevent collisions with bB games, I could just hard-code $BB in as the file type, so others could just avoid any file with $BB as the first byte.

1011153[/snapback]

 

I'm not responsible for allocating such things, but if $BB wasn't allocated to anything other than bB games, and if bB provided its own means to arbitrate among different bB games, that would be fine.

If the first byte is not $BB, the file is skipped. If it is $BB, arbitration would be done by simply reading the next 8 bytes and comparing with the specified filename. If there's a match, the file is loaded.

 

In other words, I hope to have a filesystem that works (to the programmer) much like one might expect a filesystem to work. I guess the difference is that the filenames are not stored in a directory, but instead at the beginning of the block containing the file itself. For instance, if a file is named "mygame" then the block will contain:

 

$BB, "mygame ", [119 bytes data]

 

Granted, searching through 128 blocks for the correct content might be slow, it shouldn't be THAT slow, and it should work (right?)

[supercat]

All I was going to do is implement a few commands.  I'm keeping things simple:

 

save [filename]

load [filename]

erase [filename]

 

I'd suggest having two forms of these commands. One would be as you describe. The other form would change the behavior of the commands as follows:

 

append [filename]

 

-- Save data to the specified file; even if the file exists, a new block will be created with the data

 

next [filename]

prev [filename]

 

-- Load the next or previous file with the specified name; variable "z" (or some other variable) would point to the file in question, rather than being read from it.

 

erase [filename]

 

-- Check to ensure that z points to the specified file and erase it

 

Using these facilities, it would be possible to allow the user to save multiple files and select among them.

[brpocock]

Granted, searching through 128 blocks for the correct content might be slow, it shouldn't be THAT slow, and it should work (right?)

1011204[/snapback]

 

Yes, but depending on the speed of the serial access, you might need to do some vblanks during the search. Search (n) blocks, vblank, search (n) blocks...

[brpocock]

OK. Seems like the various "conformance levels" of Supercat's proposal work well.

 

One further minor caveat: To ensure that we see application strings in the static area only when they're real: let's ask for the first 5 bytes of the app name to be ASCII7, the 6th byte ASCII7 & $80 (high bit set), just like filenames. If the high bits aren't 0,0,0,0,0,1, then we assume it's not an app ID string.

 

 

 

Static allocation blocks are statically allocated.

 

The "Scratchpad" area is available to the running app but its contents are not guaranteed to be meaningful between executions.

 

File area blocks have a leading type byte. If there is a static allocation block matching the type byte, it's checked for a 6 character type data.

 

Programs that just want to save small amounts of data can request a static allocation ID from Richard H, to guarantee no collisions.

 

Programs that just want to work with their own type of files, e.g. bB, just look for their type byte. They may, or may not, register a static block with an application type ID.

 

Programs that want to have a more "Unix-like" filer can write a metadata block with ID $00.

 

This week, time permitting, I'm planning to throw together a standard library for these functions which I've named MC_FILER. Here's the planned API:

 

At compile-time, to use MC_FILER, set MC_LEVEL=MC_FILE, or MC_FILES and include "mc_filer.s"

 

For MC_FILE and MC_FILES, the same basic rules apply. The application should define MC_FILENAME as either a RAM area of 8 bytes, or a ROM pointer if the filename is fixed. In addition, an active file# temp location will be tracked.

 

MC_SEARCH locates a filetype/name combo somewhere in the file area, and leaves it selected as the "active file." The search routine can be instructed to search for matching types and/or names based on some kind of flags (probably bits 6&7 of .a), and can be given a maximum filename length limit. EG: For CiE to find "native" files, it'll search for type $C7 (for example) and the user's name as filename; to find all available CiE files, we'd only care about type $C7; for all files on the MemCard, we just want the next block that's neither app data ($00) nor unallocated/scratched ($ff).

 

MC_READ and MC_WRITE copy from file to arbitrary RAM offsets, or from arbitrary RAM/ROM ranges to the file. There's no reason that MC_READ/MC_WRITE wouldn't work with expanded RAM on a cartridge, either. MC_WRITE can be used to change the type or filename of a file.

 

MC_SEEK selects a block in the file area to be the active block.

 

MC_CREATE allocates a new file. If the conformance level is MC_FILES then the MC_CREATE routine will also confirm the existence of, or create if necessary, an application data block with its contents copied from MC_APP_DATA in ROM.

 

MC_INFO attempts to locate information about the active file, for example, after an MC_SEARCH.

If the type byte of the active file block is $00, the type returned will be "MEMCRD". If the type byte is $ff, then "EMPTY ". Otherwise, the type string will be set to "$xx " - the hex type byte, and the static area will be checked. If the static area returns the proper high bit pattern, it will be copied into the type RAM (with the final high bit cleared). Finally, if the type is other than $00 or $ff, the file area will be searched for a matching application data block (type $00). That block will remain selected as the active file for MC_READ. If it's not found, carry will be set and an error returned in .a indicating that while the type string was found, the application data block (DESKTOP DB anyone? ) was not found. This function is only available for MC_FILES level, saving some ROM for MC_FILE level apps.

 

MC_SCRATCH removes the current active file by writing 9 $ff's to the start of the active file block. To scratch a file by name, ergo, do MC_SEARCH first!

 

Error conditions will be reported by setting carry and loading .a with an error code number, e.g. "MC_ENOFILE" or similar. If full error reporting is overkill for a particular app, the app can define a macro MC_ERRBRK to "brk" or "jmp (someplace)". The macro must be defined before include "mc_filer.s". MC_ERRBRK will not affect the following errors (they will always be returned in 'c and .a): memcard not found, file not found, and (for MC_INFO) application data block not found. (For example, in the production build of CiE, I'll probably use

 

 

To support custom screens during I/O, the routines will periodically jsr MC_WAIT_SCREEN, which should return control during the vertical blanking interval and mustn't mess with the RIOT timer.

 

Anyone wanting to access the static or scratchpad areas can just use the existing low-level drivers.

[vdub_bobby]

I've seen that proposal.  I frankly that requiring any sort of filename is massive overkill.  IMHO, the only thing the standard needs to do is allow an application to classify blocks into three categories:

 

-1- Blank

-2- Mine

-3- Someone else's

But...

Say you have two games that use the file area. Using one game, you want to save some data (or something), and there is no room left. So, you'll have to overwrite something that the other game saved.

 

But how can you tell which block you want to overwrite? Maybe you have 5 saved games from game 1, two of which you don't care about - but the other three you don't want to lose. With such a simple system there is no way to determine which block to overwrite. The only way would be if the games told you the block #; then you could compare.

 

I think Alex's email on [stella] gives some good reasons for using a filename system, though I think 8+3 might be overkill.

 

EDIT: Notably, I think using 1-byte codes for files is a bad idea, since it basically means that different carts will almost never be able to identify which games the files belong to. Using 3-byte char data for file types, at the very least, means that the user has some chance of identifying which games the files belong to.

 

After re-reading the [stella] discussion (which didn't seem so absurd to me...), I'd propose that each file begin with a 3-byte file identifier, using a 37-character alphabet (0-9, letters, underscore). Have the 7th bit of the first byte be a flag of whether or not the file is used and the 6th bit of the first byte be a flag denoting whether or not this file has a filename.

 

So: 3-character filetype is required, filenames are optional.

 

If a game wants to use filenames, at the expense of some data, great - easier for the user. If the game doesn't, that's the choice.

 

EDIT II: To avoid random data screwing things up, maybe use the 7th bit of all three filetype bytes to denote if a block is used: if all zero, block is used, else unused. Same with the filename indicator: if all three 6th-bits are zero, filename is used, else not used.

Edited February 2, 2006 by vdub_bobby

[brpocock]

Say you have two games that use the file area. Using one game, you want to save some data (or something), and there is no room left. So, you'll have to overwrite something that the other game saved

...

 

I think that's an unlikely case. Normally, cart games allocate the storage space up front. So there's only three cases here:

 

1. Everything is fine

 

2. MemCard is full, and you don't have the game that created the files. In that case, it doesn't matter which file(s) you delete.

 

3. MemCard is full, and you do have the game that created the files. In that case, switch over to that cartridge and let it review the files to choose one to delete. For example, in CiE you might see player names, in "Synthcart De Luxe" you might have longer song titles, in some other game perhaps it'd just be an indication of how many lives or what level or score ...

 

I think the basic difference is that on console games (and mainframes, not that it matters), we typically allocate storage up front, and on PC games and apps, one allocates storage "on the fly," with e.g. "File/Save As..."

 

If there was a possibility that someone might run out of MemCard storage during game play, then we'd have more trouble, but I think that's unlikely to occur.

 

To quote the old Stella discussion,

 

I prefer text to codes because:

 

- No list of codes has to be maintained.

 

I think that's a de facto fallacy, though. No matter what, someone will have a collision unless there's an effort to keep a registry. Yes, with 3 bytes instead of 1 the odds are reduced greatly, in some respects, but it also introduces a psychological aspect. For example, in PC-DOS/Windows, "DOC" is used by (at least) MicroSoft Word (several mutually-unintelligible dialects), MicroSoft WordPad, WordPerfect, and plain text files. On the other hand, Apple ProDOS used one-byte type codes, and still had collisions between apps, even though Apple maintained an official registry service... which is why MacOS and (I think?) SOS went to using 4-byte creator codes, and MacOSX recommends appending extension types like ".iMovieProject" ... If we assume that 2 developers are working on save game functionality without some means of communications, it's a safe bet that "SAV" or "GAM" will be used by two different games in fairly short order.

 

My initial concept of using 8×8 icons for file types was based on the idea that bitmaps are far less likely to "collide" than text, and 2**64 (18 quadrillion, US = 18 thousand billion, commonwealth) is a larger search space than 2**15 (3 ASCII characters ~ 32,768) or 2**8 (254 "usable" one byte type codes).

 

But, having Richard H. deal with allocating type ID's = static allocation blocks provides a neat answer to any possible namespace collisions, I think, and also provides for some great answers to other game-related issues.

 

"System" settings can go in the static area... (e.g. PAL v. SECAM flag? Turn off sound effects?) "Per-Player" settings or "per-document" settings or whatever is appropriate go into individual files. And apps that handle files can, optionally, provide icons for their files.

 

Plus, not that it matters greatly, but this is similar to the way files are handled on machines like the GCN that face the same division between ROM and storage.

 

Notably, I think using 1-byte codes for files is a bad idea, since it basically means that different carts will almost never be able to identify which games the files belong to.

 

Actually, Supercat handled that case by giving a 6-byte user-visible application string in the static allocation area, and an additional (but optional) expanded application info area for a user-visible file icon at 8×7px and what amounts to an entire splash screen bitmap.

 

Hopefully, whether using some common library like MC_FILER or "home-grown" routines specific to one particular game, when a list of "foreign" or "all" files is shown to the user, the 6-char type (at least) will be displayed, and preferably, also the icon (at least the 8×7 version).

 

Reviewing the old [stella]list discussions ... not to trivialize the thought that went into it, but: I doubt anyone is really going to do Huffman coding on filenames and such, and even having a filename could be meaningless in many cases. Paul Slocum's & Supercat's idea of type bytes + optional filenames seem fair. If we could come to an agreement on allowing variable-length filenames with either zero byte termination or high bit set on last character, that'd be great.

 

The amount of work required to support such a scheme is pretty minimal, and different carts can choose to participate at various levels of compliance (just types, types+6char id's, types+id's+full app info), based on how critical file management is, how much ROM space the app can spare, &c.

[supercat]

Actually, Supercat handled that case by giving a 6-byte user-visible application string in the static allocation area, and an additional (but optional) expanded application info area for a user-visible file icon at 8×7px and what amounts to an entire splash screen bitmap.

1011611[/snapback]

 

It's hard to predict how popular the memcard is going to be with developers. If everyone who feels like trying his hand at doing a bB game gets a static allocation they might run out fairly quickly. But it would also be unfortunate to have people go to great lengths to avoid using up the static area and then discover that in the next twenty years only the first 30 slots ever got allocated.

 

If the design of a game is such that it will naturally benefit from a static area, the address of that area is a good unique way of identifying the game. If a programmer wants to avoid screen blanking while looking for files, using a short identifier may be much faster than a long one, though I don't know exactly where the speed cutoff would be between checking one page per frame and checking one page per 1.5 frames (assuming checking is done in both overscan and vblank).

 

Perhaps it might not be a bad idea to specify something akin to Internet class A, class B, etc. address spaces, with the distinction being relevant only to applications that want to show the user something about who created what files.

 

Basically, the issue such a program would need to address would be how many bytes at the start of the data page must match the start of the "program info" page. I'd suggest the convention:

 

08-BF : Match 1 byte (1 byte ID)

C0-CF : Match 2 bytes (12-bit ID)

D0-CF : Match 4 bytes (28-bit ID)

[brpocock]

Perhaps it might not be a bad idea to specify something akin to Internet class A, class B, etc. address spaces, with the distinction being relevant only to applications that want to show the user something about who created what files.

 

Works for me.

 

[vdub_bobby]

Enough already with the Huffman thing. Yeah, it is kind of a dumb idea, but one person, in their only contribution to the entire discussion, brought it up and it isn't like anybody else thought it was a good idea.

 

Do you want somebody someday to reread this thread and dismiss it in its entirety just because of my dumb contributions?

 

Anyhoo.

 

I don't think tying the files to the static allocation area is necessarily the best idea, because (A) if we do ever run out of static allocation area then, boom, nobody can ever use the file area anymore either. Whoops! Also, (B) that could also contribute to the static allocation area being used faster, since everyone who wants to use the file area must, by necessity, gobble up a static block.

 

Relatedly: Assigning a single-byte type code to every bB game is probably not a good idea because then more than one bB game can't coexist on the Memcard! You might say, they can each assign different filenames so it doesn't matter that they all share a type code, but under this system isn't the lookup for the filenames in the static area? And if all bB games share the static area, won't they all overwrite it with their own information?

 

IMO a system that makes the "free" blocks unusable when the static blocks are used up is not a good system. The free blocks should be kind of a failsafe for the static blocks.

 

Also...

It's easy to say that a cart will allocate it's Memcard memory upon bootup, but what if...

-a game supports unlimited game saves during gameplay?

-a game saves files of varying length during gameplay?

-a user doesn't dig out his Memcard until he wants to save a game, in midgame play? Now, you could require him to plug it in when he starts his game, so that problem would go away, I suppose.

 

As an example, I was toying with, for the SC contest, writing an Arkanoid port with a level editor, and allowing you to save levels to the Memcard.

 

For something like that, how would you allocate all the memory you would need upon bootup? Allocate to some upper limit and then if the user reaches that limit, tell him to cycle the power? That's a chintzy solution.

 

EDIT: The icon idea is pretty cool, by the way. Though I still think a 3-character code for every file plus an optional filename/icon is the best way to go. Doesn't require a game to read anything additional from the static area. Doesn't require a game to use a static area if it doesn't want/need to. And I don't see the duplicate name problem being such an issue as you do. If the filetype codes are understood as being primarily identifiers of the game they are associated with and only secondarily as identifiers of the file, I don't see everybody using SAV or GAM for their filetype codes. Plus, the 2600 dev scene, even including all the bB coders, isn't exactly huge. It shouldn't be too hard to find out if the filetype code you chose overlaps somebody else's.

 

EDIT II: I think the scratch-pad idea is also a very good idea.

Edited February 3, 2006 by vdub_bobby