Distella configuration files

[Omegamatrix]

Thinking about it now, you know what the ultimate would be? Being able to pass mnemonics into the debugger for ram that I figure out in a disassemble. I know there's not a lot of space in the debugger, but the names don't have to be that long and it would help the user follow the code. So instead of this:

 

LF401:
    lda    $83
    and    #$07
    bne    LF40C
    lda    $9A
    sta    COLUP0
...

 

 

You could do this by choice:

 

LF401:
    lda    frameCount
    and    #$07
    bne    LF40C
    lda    bearColor
    sta    COLUP0
...

 

 

 

 

 

Also the better that Stella can disassemble automatically and correctly, the better it gets overall. However if you ever wanted to add support for hard-coded configuration of old era games... Well almost all the games have been disassembled now and it would just be a matter of rounding up all the configuration files, verifying them, and matching them to the correct MD5. This is a separate idea from passing a configuration file. Let me explain.

 

 

I envision this as an option within the debugger that the user can turn on or off. When turned on, Stella would first check if there is a config file (not passed by the user, this configuration would be part of the Stella code somehow), and if the configuration file is found for the rom it's used, otherwise Stella proceeds with auto disassembly.

 

 

I can help you with getting all the configuration files if you find this idea interesting. And again it is a separate idea from being able to pass configuration files. I still would like to be able to pass a configuration file that I write. I guess the way to look at this separate idea is to see it as a catalog of data that the user can choose to use/not use.

[+stephena]

Must resist the urge ...

It's tempting isn't!! I'd hate to steal you away from Stella though. A lot more people use Stella then Distella, and 3 months would be huge. But if you ever get bored with Stella and want to switch it up for a bit...

Yes, I just have to get this out of my mind for now, or else I'll pick it up and start working on it.

 

As for the remaining comments, I'll take them in pieces:

As for passing the configuration files... my original idea long, long ago, in a galaxy far, far away was to "lock" some old era games into an accurate disassembly. It will clearly show me the gfx, data, and code that I told it to in the configure file, and it would look somewhat like how a Distella output would. However if you ever wanted to add support for hard-coded configuration of old era games... Well almost all the games have been disassembled now and it would just be a matter of rounding up all the configuration files, verifying them, and matching them to the correct MD5. This is a separate idea from passing a configuration file.

This is relatively easy to do, I think. I could either integrate it into ROM properties or (more likely) just create a database of distella commands for ROMs, similar to the cheatcode database (which doesn't actually exist right now, but the infrastructure is there). So we can move past this one for now, as it's relatively easy to do, once I figure out the format to use for the config file.

 

For the homebrewers (or hackers), it helps to have the address printed right after the data or gfx byte in the debugger. With that you can quickly pinpoint and determine if you program is loading the byte you want or (one above/below in the table, oops!!). If the GFX command can be used to make the graphics look like it does in Distella, then visually it would help you locate where you are in the code quicker by separating it from DATA. Plus it looks better and helps all those graphics hackers too.

Yes, this of course is the goal, but for now we won't talk about how the output will look. The look can be tweaked later; I want to nail down how to get the settings into Stella in the first place.

 

I was especially going to use the configuration file to take care of the BIT, NOP, or CMP instructions that are just being used to skip a byte or two since Stella couldn't handle it at that time. If you want to know how bad these can be, take a look at the last bank of Pete Rose Baseball. That is by far the longest configuration file I've ever done, and it was mostly due to all the BIT instructions in there. When you are trying to follow the logic in the debugger it is a real mess because the code keeps jumping into those BIT instructions, etc...

So you're saying that Stella now supports this, and it's no longer a problem?

 

Thinking about it now, you know what the ultimate would be? Being able to pass mnemonics into the debugger for ram that I figure out in a disassemble. I know there's not a lot of space in the debugger, but the names don't have to be that long and it would help the user follow the code.

Stella already supports loading from a .sym file, generated from DASM. In this .sym file, you can have labels for any address you like. In fact, the 3.2 release added support for user labels in the disassembly again. User labels are exactly what you refer to as 'mnemonics'. There are several commands to access and modify these labels, and they can be saved into a .stella file for future use. So that functionality is already there, and doesn't need to be added to anything Distella-related.

 

And BTW, as for there not being enough space in the debugger/disassembly, you are aware you can make the debugger window bigger??

 

I envision this as an option within the debugger that the user can turn on or off. When turned on, Stella would first check if there is a config file (not passed by the user, this configuration would be part of the Stella code somehow), and if the configuration file is found for the rom it's used, otherwise Stella proceeds with auto disassembly. I can help you with getting all the configuration files if you find this idea interesting. And again it is a separate idea from being able to pass configuration files. I still would like to be able to pass a configuration file that I write. I guess the way to look at this separate idea is to see it as a catalog of data that the user can choose to use/not use.

This is an interesting idea, and doesn't seem too hard to do. However, I'm still unclear how to best present this choice to the user. Here's the approach I'm thinking so far:

 

1) There's a main list of config files that has been collected. Whether this is hard-coded or stored in a file is irrelevant right now. The point is, it is global.

 

2) If a global config file exists, use it. If it doesn't exist, then we have the situation the same as now, and disassembly occurs the best way it can.

 

3) If a user config file exists, it overrides the global one. This takes care of wanting to use your own config files. Note that if you modify the global config file for a particular ROM, you're essentially creating a user config file. This way, the global configs are never changed, only the user ones. Note that using any of the new distella-related commands within the debugger prompt (code, data, gfx) will automatically create a user config file, which can be saved with the 'savecfg' command (which I haven't written yet). There will also be 'loadcfg' and 'listcfg', whose functionality is obvious.

 

Note that while this scheme seems complicated, it's really the only way to ensure that user settings override global ones. Note that the ROM game properties work exactly the same way (there's an internal properties database, which if you modify in any way can be saved into a personal stella.pro file).

[Nukey Shay]

As for passing the configuration files... my original idea long, long ago, in a galaxy far, far away was to "lock" some old era games into an accurate disassembly. It will clearly show me the gfx, data, and code that I told it to in the configure file, and it would look somewhat like how a Distella output would. However if you ever wanted to add support for hard-coded configuration of old era games...Well almost all the games have been disassembled now and it would just be a matter of rounding up all the configuration files, verifying them, and matching them to the correct MD5. This is a separate idea from passing a configuration file.

This is relatively easy to do, I think. I could either integrate it into ROM properties or (more likely) just create a database of distella commands for ROMs, similar to the cheatcode database (which doesn't actually exist right now, but the infrastructure is there). So we can move past this one for now, as it's relatively easy to do, once I figure out the format to use for the config file.

But how to label the memory segments? All data as data...or seperated into specific groups...bitmap GFX, data tables which have nothing to do with bitmaps, color tables/memory locations for easy color hacking, and vector lists (which make the program non-relocatable until it's data is altered into labels). Of the ones that I've done, it's pretty easy to come up with a "memory map" for each one...except for that last bit - which almost requires full reverse-engineering for the complex ones. But as Omega stated, pretty much everything exists with those parameters mapped out by somebody here, subscribed to the Stella mailing list, etc. I could start rounding mine up and fixing them if I knew how the data should be divided. IMO there's no need to stick to Distella's 3 catagories, because it itself is so outdated.

[+stephena]

As for passing the configuration files... my original idea long, long ago, in a galaxy far, far away was to "lock" some old era games into an accurate disassembly. It will clearly show me the gfx, data, and code that I told it to in the configure file, and it would look somewhat like how a Distella output would. However if you ever wanted to add support for hard-coded configuration of old era games...Well almost all the games have been disassembled now and it would just be a matter of rounding up all the configuration files, verifying them, and matching them to the correct MD5. This is a separate idea from passing a configuration file.

This is relatively easy to do, I think. I could either integrate it into ROM properties or (more likely) just create a database of distella commands for ROMs, similar to the cheatcode database (which doesn't actually exist right now, but the infrastructure is there). So we can move past this one for now, as it's relatively easy to do, once I figure out the format to use for the config file.

But how to label the memory segments? All data as data...or seperated into specific groups...bitmap GFX, data tables which have nothing to do with bitmaps, color tables/memory locations for easy color hacking, and vector lists (which make the program non-relocatable until it's data is altered into labels). Of the ones that I've done, it's pretty easy to come up with a "memory map" for each one...except for that last bit - which almost requires full reverse-engineering for the complex ones. But as Omega stated, pretty much everything exists with those parameters mapped out by somebody here, subscribed to the Stella mailing list, etc. I could start rounding mine up and fixing them if I knew how the data should be divided. IMO there's no need to stick to Distella's 3 catagories, because it itself is so outdated.

There's nothing specific in the Distella code that says one can only use CODE, DATA or GFX sections; it's just all that was included when the app was written, I guess. With the modified Distella built into Stella, it's easy to add more categories, and have the app output lines differently based on category.

 

Like I was saying above, this is simply a tweak on the 'directives' that Distella can understand, and the output that it can generate. My main goal at this point is nailing down the file format. I'm thinking something like this:

 

[0]
ORG ...

multiple lines of DATA/CODE/GFX
...
+ perhaps some other directives

[1]
ORG ...

multiple lines of DATA/CODE/GFX
...
+ perhaps some other directives

.
.
.

And so on, with basically a different section for each bank. So you see, adding directives for fine-tuning different types of data is easy, and not really my focus right now. Problem is, once we go down that road, the actual standalone Distella app will have to be fixed too. I'm open to discussing what new categories we can have, and how its respective output should look. But I feel that I'm stepping into maintaining Distella with every suggestion I make, and unfortunately I really don't have the time to do it right now (although it would be a very interesting project to take on ...).

[+stephena]

OK, you've gone and done it. Now I have to take a look at reworking Distella too I'll give it two weeks to see how far I can get, but I'd say I'm 70% of the way there already (with the recent additions to Stella). My goal is to have the new Distella in C++ (already done), and be interchangeable with the Stella code (ie, I should be able to take the DiStella.cxx class and drop it into both Stella and the external Distella application, without changes). The only way I can find the time to work on both projects is if the code is exactly the same.

 

So, I have to add back a few things I removed. First off, A7800 support will not be added back, at least not by me. It doesn't apply to Stella, and I don't have the time to test it anyway. Secondly, the config file syntax will obviously have to change. This means older config files will have to be updated. Since all older config files are for a single bank, it will probably mean just adding a '[0]' to the top of the file, indicating that the following commands apply to bank 0.

 

Longer term, my goal is for Stella to generate config files, using knowledge that it gains during emulation and disassembling within the debugger. From that point, it would be easy to have Stella spit out the ASM file too, but having it generate the config file is sufficient.

 

The major benefit of doing things this way is, as I've said, I can work on Distella without taking over full maintainership or neglecting Stella. If I do it correctly, creating an up-to-date Distella release will just be a natural result of my work in Stella.

 

At the same time, I'll need feedback and some help. So any suggestions are welcome.

 

Finally, I can't start this until next week, as my laptop is at work, and I've been off sick for the past few days.

[Omegamatrix]

I hope you are feeling better. I myself was moving all weekend and had no access to internet. Not very fun at all.

 

 

I was especially going to use the configuration file to take care of the BIT, NOP, or CMP instructions that are just being used to skip a byte or two since Stella couldn't handle it at that time.

So you're saying that Stella now supports this, and it's no longer a problem?

 

No, it's still a problem, but I figured it'd be taken care of either by passing a configure file, or some new code written by yourself where the disassembler detects a jump or branch into a bit instruction, etc...

 

 

 

Here's the approach I'm thinking so far:

 

1) There's a main list of config files that has been collected. Whether this is hard-coded or stored in a file is irrelevant right now. The point is, it is global.

 

2) If a global config file exists, use it. If it doesn't exist, then we have the situation the same as now, and disassembly occurs the best way it can.

 

3) If a user config file exists, it overrides the global one.

 

 

This priority system is bang on.

 

 

 

Note that if you modify the global config file for a particular ROM, you're essentially creating a user config file.

I'm wondering how you are picturing the user having access to the configuration file? Specifically can they only see it through the debugger, or will there be a folder containing all the global configuration files? A folder would be nice as I could load directly load the configuration file into my text editor, and save it back in the folder. Of course you should be able to change or create configuration files in the debugger prompt too.

 

 

 

Updating or creating a file directly in the debugger would find the most use for me in the form of partial configuration files. I would like it if I could quickly type in a line or two, and the graphics from $B000 - $B0FF in bank 2 is shown as well, gfx. This would be used in a debugging situation where your gfx is coming out garbled, and you are trying to nail down if it is because you're loading a few bytes higher or lower then you should be. The idea of parital configuration files is you can lock certain regions to how you want them, and Stella well auto disassemble the rest.

 

 

 

The natural extension of that is being able to write complete configuration files in the debugger as well, by choice. That would become very attractive for me if it was a dynamic interaction between typing in a new configuration line and Stella's code window automatically updating as you go on from line to line. Basically the hard part of trying to write a configuration file is getting the exact addresses between code/gfx/data correct. So as I modify the configuration file Stella automatically updates the code, and I can immediately tell whether I have the correct address range for that bit of code or gfx, etc...

[+stephena]

I was especially going to use the configuration file to take care of the BIT, NOP, or CMP instructions that are just being used to skip a byte or two since Stella couldn't handle it at that time.

No, it's still a problem, but I figured it'd be taken care of either by passing a configure file, or some new code written by yourself where the disassembler detects a jump or branch into a bit instruction, etc...

OK, I can look into this later.

 

Note that if you modify the global config file for a particular ROM, you're essentially creating a user config file.

I'm wondering how you are picturing the user having access to the configuration file? Specifically can they only see it through the debugger, or will there be a folder containing all the global configuration files? A folder would be nice as I could load directly load the configuration file into my text editor, and save it back in the folder. Of course you should be able to change or create configuration files in the debugger prompt too.

Both approaches are possible. It's extremely easy to have Stella auto-load a config file (based on ROM filename from the properties database). And once the config is in Stella, modifying it from the debugger prompt would create a new in-memory version. Then saving would overwrite the external file. Or modifying it outside Stella and then issuing a load within Stella to get the updated version. All of this is extremely easy, and some of it is already in SVN now.

 

Updating or creating a file directly in the debugger would find the most use for me in the form of partial configuration files. I would like it if I could quickly type in a line or two, and the graphics from $B000 - $B0FF in bank 2 is shown as well, gfx. This would be used in a debugging situation where your gfx is coming out garbled, and you are trying to nail down if it is because you're loading a few bytes higher or lower then you should be. The idea of parital configuration files is you can lock certain regions to how you want them, and Stella well auto disassemble the rest.

I just committed code today that does this. I added code, data and gfx commands to the debugger prompt, that do exactly what you expect. Type 'gfx f000 f100', and the currently active bank will be re-disassembled with those commands. This is dynamic and immediate; issue the command, and the results are immediately shown.

 

The natural extension of that is being able to write complete configuration files in the debugger as well, by choice. That would become very attractive for me if it was a dynamic interaction between typing in a new configuration line and Stella's code window automatically updating as you go on from line to line. Basically the hard part of trying to write a configuration file is getting the exact addresses between code/gfx/data correct. So as I modify the configuration file Stella automatically updates the code, and I can immediately tell whether I have the correct address range for that bit of code or gfx, etc...

Yes, this is possible right now. I'm currently working on the loadconfig, saveconfig, and listconfig commands, which should be self-explanatory. But this part is easy; actually passing the options to Distella and getting it to re-disassemble was the hard part (and it's now complete).

 

Adding config file options (what I call 'directives' within Stella) is also being done 'smart'. That is, if you already have a DATA range from f000 - f100 and you then add a GFX range from f020 - f050, Stella will automatically split the data range, resulting the 3 ranges: DATA, then GFX, then the remaining DATA. I'm still working on this part, but expect to have it done tomorrow or Wednesday.

 

I'm also optimizing the saveconfig command to use all possible knowledge. That is, when you save a config file, it shouldn't just save the directives that exist at that point; it should also include results from auto-determination. For example, $f81a is the start of a scanline in River Raid, which the disassembler automatically finds. So saving a config file should also include that directive, even if it wasn't present in the original list of directives. I expect to have this functionality done by the end of the week.

[Omegamatrix]

Adding config file options (what I call 'directives' within Stella) is also being done 'smart'. That is, if you already have a DATA range from f000 - f100 and you then add a GFX range from f020 - f050, Stella will automatically split the data range, resulting the 3 ranges: DATA, then GFX, then the remaining DATA.

 

Very clever idea. I like this one a lot.

 

 

I'm also optimizing the saveconfig command to use all possible knowledge. That is, when you save a config file, it shouldn't just save the directives that exist at that point; it should also include results from auto-determination. For example, $f81a is the start of a scanline in River Raid, which the disassembler automatically finds. So saving a config file should also include that directive, even if it wasn't present in the original list of directives. I expect to have this functionality done by the end of the week.

 

I see where you're going with this. It helps you complete the configure file as best it can.

 

How does priority work in this situation? If I declare something as gfx/data and Stella detects a jump into there, would it change it to code automatically with a smart split?

[+stephena]

I see where you're going with this. It helps you complete the configure file as best it can.

 

How does priority work in this situation? If I declare something as gfx/data and Stella detects a jump into there, would it change it to code automatically with a smart split?

The directives have the same precedence as they would in Distella. I forget what they are right now, but you can look at the Distella readme to find out. Remember, all this stuff we're doing is just launching Distella behind the scenes, so it acts exactly the same way as standalone Distella.

 

But I agree that in your example, it would be nice to have the automatic determination override the config file. However, Distella was written to work the other way around; config options set by the user override the determination itself. And I'm not sure how to work around that one. It's almost as if we're saying directives have priority, unless the automatic determination conflicts with it, in which case the auto-determination has priority. Stella provides more info to Distella than a standalone, static run, so perhaps this should be the case?

 

Arrgh, I need to think about this a little more ...

[Omegamatrix]

But I agree that in your example, it would be nice to have the automatic determination override the config file. However, Distella was written to work the other way around; config options set by the user override the determination itself. And I'm not sure how to work around that one. It's almost as if we're saying directives have priority, unless the automatic determination conflicts with it, in which case the auto-determination has priority. Stella provides more info to Distella than a standalone, static run, so perhaps this should be the case?

 

Arrgh, I need to think about this a little more ...

 

 

I say just leave it as is. The configure file having priority makes sense to me. It's easy to understand. Plus it gives the user more control to manipulate the output in the code window to how they want. For some strange reason they might decide to change all the code to data except one small part that they are trying to debug. Might be a bad example really... the important part though is people will try to do some strange things, and in the end the user should have the final say with their configuration file. Less headaches that way, and total control.

[+stephena]

OK, I've now implemented the 'listconfig' debugger command, which lists both user-defined directives as well as the complete results of disassembling a bank (using all the extra knowledge that Stella has, ie cached entry points). Still TODO is the loadconfig and saveconfig commands, but they're pretty easy by comparison.

 

Maybe I can provide test builds for you guys to try out? I guess Windows versions would be best??

[Omegamatrix]

Maybe I can provide test builds for you guys to try out? I guess Windows versions would be best??

Yep, windows please.

[+stephena]

Maybe I can provide test builds for you guys to try out? I guess Windows versions would be best??

Yep, windows please.

OK, I didn't get this message until this morning, so I'll have to wait until later today, when I get access to a Windows VM at home.

[+stephena]

OK, here's a test version for Windows (32 and 64 bit): Stella-3.3_test1-windows.zip. This is on my personal server, so please don't spread it around too much.

 

There are new commands in the debugger prompt: code, data, and gfx. These work exactly as in a Distella config file. They have the same priority as in Distella as well, and add Distella options (henceforth called 'directives'). They modify the current bank; if you want to modify another bank, you need to switch to it first. To remove a directive, type the same command again. For example, if 'GFX f000 f010' has been defined, typing the same command again will undefine it. The directives are smart, in that the ranges are automatically split and merged as required. So if you have 'DATA f100 f200', and you want a gfx section in the middle, just type 'GFX f150 f180'. Stella will automatically split the range into three pieces. And if you later define the middle range as data, the ranges will be merged together.

 

There are a few other new commands: listconfig, loadconfig, and saveconfig. The latter two are just stubs for now, and still need to be implemented. The first one lists all Distella config items set by the user, and the resulting directives after disassembling the bank(s). Typing 'listconfig x' will show info for bank x, while leaving off the bank will show info for all banks. The reason for listing the directives separately is that while user directives will influence the results, they aren't the whole story; previously cached entry points will also be taken into account. So the longer you run the debugger and step through code, the more accurate the directives will become. When saveconfig is eventually implemented, it will save the second list of directives (the ones covering the entire address space of all banks).

 

Finally, note that there's tab completion in the debugger prompt, so you don't have to type out the whole command(s) each time

[Omegamatrix]

Seriously, I think this is the coolest thing I've seen in a long time. I've waited so long to be able to see GFX in the debugger!!

 

 

First thoughts (and this might be a long post, sorry).

 

1) the smart split worked perfectly

2) I love seeing gfx displayed in the debugger

3) I love how the debugger code updated itself on the fly when I entered a new line in the configuration

- When I type a new line for the configuration it updates itself, but the problem now is that the default is for the code window to snap back to the last line done/drawn by Stella. This is a problem only because I may be at the other end of the rom working out the disassembly. So I have to scroll all the way back to the end to view the last changes I did. It would be more user friendly if the screen could stay wherever I had it scrolled too, and still updated itself.

4) With the limited width of the debugger, it'd be nice to also have a data command that would show the DATA line by line (like the GFX command does). This is what Nukey suggested before, and I think it'd look really nice. The only visual difference between it and the GFX would be DATA wouldn't display the gfx preview. This would be great as the default for DATA (in the debugger)

5) The present DATA command is still useful for garbage code. I don't want to loose it!! I just want to have both options.

6) With the GFX now displayed in the debugger (and this is huge), the option to directly hack the gfx is there. GFX artists will love this option in Stella, as they can open the rom, see the sprite, and update the rom on the fly.

- However, the same issue occurs with Stella snapping back to the last line drawn when it updates itself. I have a preview with what I did here in Frostbite:

 

 

You can see I was messing around adding configuration lines, and then I directly hacked $FC0C. It worked, but the code updated and snapped way back to the beginning of the rom.

- speaking of directly hacking, I love the look of having HEX values displayed on the right edge because of the nice clean look. However I feel this would be a nusciance to the GFX artists, and they would prefer to be able to directly edit the GFX in binary so they could update it bit by bit. Is there anyway to make it output binary in the far right portion of the window for the GFX command only? Leave data and normal code defaulted to hex to help distinguish it quickly while scrolling.

 

Also I'm asking for that to be default because I'm thinking of other people here. I myself really do prefer HEX, but I do believe the GFX artists would find it much more useful in binary for the GFX. The ultimate solution for me would be to have the best of both worlds. Make the default binary for gfx, but somewhere, somehow put in a global option in Stella that I can switch this back to HEX. I was hoping somewhere in the UI settings. There already seems to be some debugger and miscellaneous options in there.

 

 

 

Finally, this is all amazing work. I'm looking forward to start feeding you configure files that can be added to Stella, and I know Nukey is too. This is really KICK ASS.

[+stephena]

Seriously, I think this is the coolest thing I've seen in a long time. I've waited so long to be able to see GFX in the debugger!!

Yes, I thought it was cool too, if I do say so myself Good to hear about what worked, so I'll only address the problems:

 

When I type a new line for the configuration it updates itself, but the problem now is that the default is for the code window to snap back to the last line done/drawn by Stella. This is a problem only because I may be at the other end of the rom working out the disassembly. So I have to scroll all the way back to the end to view the last changes I did. It would be more user friendly if the screen could stay wherever I had it scrolled too, and still updated itself.

Yes, I noticed that too. Really annoying, and something I will have to fix. The cursor doesn't have to jump until the PC changes, not after a modification. Or perhaps it shouldn't jump at all. One can always press 'Space' to snap back to the PC line.

 

With the limited width of the debugger, it'd be nice to also have a data command that would show the DATA line by line (like the GFX command does). This is what Nukey suggested before, and I think it'd look really nice. The only visual difference between it and the GFX would be DATA wouldn't display the gfx preview. This would be great as the default for DATA (in the debugger)

Yes, we spoke about this before. I assume you mean this would be the 'BDATA' command? For now, I'm just concerned with getting everything working with the 3 commands that Distella supports. We can extend it later for more types.

 

speaking of directly hacking, I love the look of having HEX values displayed on the right edge because of the nice clean look. However I feel this would be a nusciance to the GFX artists, and they would prefer to be able to directly edit the GFX in binary so they could update it bit by bit. Is there anyway to make it output binary in the far right portion of the window for the GFX command only? Leave data and normal code defaulted to hex to help distinguish it quickly while scrolling.

I was going to do this, but you mentioned you preferred hex

 

Also I'm asking for that to be default because I'm thinking of other people here. I myself really do prefer HEX, but I do believe the GFX artists would find it much more useful in binary for the GFX. The ultimate solution for me would be to have the best of both worlds. Make the default binary for gfx, but somewhere, somehow put in a global option in Stella that I can switch this back to HEX. I was hoping somewhere in the UI settings. There already seems to be some debugger and miscellaneous options in there.

I can easily have it show binary, and also support hex. It can look at the size of the string and convert accordingly. As for what comes up by default (and changing the default), I guess a config option could be added.

 

EDIT: On a related note, the disassembly still isn't as complete as it could be. Right now, Stella provides Distella with what I call 'entry points'. That is, program counter addresses encountered in the debugger. This helps Distella produce more accurate CODE output, as it often finds code that can't be found from a static analysis alone. River Raid is the best example. Its frame starts at $f81a, and a whole block of code follows. However, using external Distella skips this entire range, and marks it all as DATA.

 

As you run the debugger (step, trace, advance, etc), more PC entry points are found and added as hints for Distella, resulting in more accurate disassembly the longer you run it. However, as a consequence of this, you must be running from within the debugger. Put another way, PC entry points are only added in the debugger, not during normal emulation. A better way to do this, and something I'll work on for the next release, is to mark CODE sections from the emulation itself. That way, any time an address is in the PC, it will be marked as CODE. This is obviously much better, as you won't have to step in the debugger to find all possible code paths. The only thing stopping me from doing this (other than time) is making sure it doesn't adversely affect emulation running time (the casual user shouldn't be inconvenienced by debugger features they may never use).

[Omegamatrix]

The cursor doesn't have to jump until the PC changes, not after a modification. Or perhaps it shouldn't jump at all.

 

I definitely like it jumping back after the PC changes, because that is the main way I get around when I'm looking at another place in the rom. I'll press step or scanline followed by the rewind button, and that way I don't have to search for where it was. Stopping it from jumping after a modification though is something completely different, and would be a great fix.

 

 

 

 

Is there anyway to make it output binary in the far right portion of the window for the GFX command only? Leave data and normal code defaulted to hex to help distinguish it quickly while scrolling.

I was going to do this, but you mentioned you preferred hex

 

I do, I really do... but I can't justify what I personally like above what I think will ultimately make Stella more attractive for the masses. As long as I can get a switch put in somewhere where I can default it back to how I like it... well I'm more than thrilled. I'm also hoping I will just have to flip this switch once and It'll stay like that until I change it again.

 

 

For Distella, a simple switch would also work well to convert between hex and binary GFX. I mean I already am typing -pasfc in the command prompt for every disassembly I do, so I don't mind an extra character like a h in there to specify hex or anything.

 

A better way to do this, and something I'll work on for the next release, is to mark CODE sections from the emulation itself. That way, any time an address is in the PC, it will be marked as CODE. This is obviously much better, as you won't have to step in the debugger to find all possible code paths. The only thing stopping me from doing this (other than time) is making sure it doesn't adversely affect emulation running time (the casual user shouldn't be inconvenienced by debugger features they may never use).

 

Yeah, slowing down emulation speed would be disastrous. Well, at least computers are getting more processing power all the time. I do believe that this is how FCEultra for the Nintendo works, by making parts of the rom as code or data as it is being played. So I have hope, and I agree it would be superior for auto-disassembly, which IMHO will be the greatest strength of the Stella disassembler. I mean if I can play a game for a little bit, and then go into the debugger and have it spit out a configuration file, well that's just the cat's ass right there.

[+stephena]

The cursor doesn't have to jump until the PC changes, not after a modification. Or perhaps it shouldn't jump at all.

 

I definitely like it jumping back after the PC changes, because that is the main way I get around when I'm looking at another place in the rom. I'll press step or scanline followed by the rewind button, and that way I don't have to search for where it was. Stopping it from jumping after a modification though is something completely different, and would be a great fix.

OK, this is now fixed. The cursor is only moved when the PC changes. And a nice consequence is that GFX can be changed, you can do a frame advance to see the results, and since the PC is the same at the end of every frame, the position won't change.

 

Is there anyway to make it output binary in the far right portion of the window for the GFX command only? Leave data and normal code defaulted to hex to help distinguish it quickly while scrolling.

The default is now to show binary data for GFX sections. Still TODO is allow the default to be set to either binary or hex. BTW, you can enter hex anywhere if you prefix it with a '%' symbol. The same is true for decimal if prefixed with '#'. It was also supposed to be possible to enter binary by prefixing with '\', but there's a bug that prevents typing the '\' symbol

 

So I have hope, and I agree it would be superior for auto-disassembly, which IMHO will be the greatest strength of the Stella disassembler. I mean if I can play a game for a little bit, and then go into the debugger and have it spit out a configuration file, well that's just the cat's ass right there.

I have some ideas on how to accomplish it without slowdown, but for now I'm going to concentrate on directive handling (and implement load and save).

 

I've rethought how config/directive files will work. There will be directory where such files can be stored, and these will be global. That is, they're the ones that are used if no other ones exist. This allows us to create a default set of files, and zip them up and distribute from Stella separately.

 

Then we'll have user files. These are located at the same location as the ROM file itself (ie, in the same directory). If one of these exists, it will always override the global one. Saving a new file will write to this location, not the global one. Overall, my reasoning is that global files should be read-only, and user ones be read-write. And realistically, probably very few people will be creating user files, and just relying on global files that we come up with.

[Omegamatrix]

OK, this is now fixed. The cursor is only moved when the PC changes. And a nice consequence is that GFX can be changed, you can do a frame advance to see the results, and since the PC is the same at the end of every frame, the position won't change.

 

Awesome! quick too. I thought this might be a real difficult thing to accomplish, but it doesn't seem to bad.

 

 

The default is now to show binary data for GFX sections. Still TODO is allow the default to be set to either binary or hex.

 

Thank you! And again, I'm amazed at your speed here!

 

 

BTW, you can enter hex anywhere if you prefix it with a '%' symbol. The same is true for decimal if prefixed with '#'. It was also supposed to be possible to enter binary by prefixing with '\', but there's a bug that prevents typing the '\' symbol

 

I never knew, or thought of this. I'm so used to '%' being the default for binary, and '$' being the default for hex.

 

 

I have some ideas on how to accomplish it without slowdown, but for now I'm going to concentrate on directive handling (and implement load and save).

 

I've rethought how config/directive files will work. There will be directory where such files can be stored, and these will be global. That is, they're the ones that are used if no other ones exist. This allows us to create a default set of files, and zip them up and distribute from Stella separately.

 

Then we'll have user files. These are located at the same location as the ROM file itself (ie, in the same directory). If one of these exists, it will always override the global one. Saving a new file will write to this location, not the global one. Overall, my reasoning is that global files should be read-only, and user ones be read-write. And realistically, probably very few people will be creating user files, and just relying on global files that we come up with.

I agree with all of this. I think the greatest compliment will be the general public will know it works, without thinking to much as to how or why. That's the way is should be. It should simply just work well. Making the code window not jump when you edit one of the gfx values is an prime example of making it work well. The end user will never know you fixed that, but they won't be thinking about how annoying it is jumping around all over the place. Hopefully they will just be thrilled that you can edit values directly. I know I was.

 

Those that really want to make their own files will figure it out. I know I plan on using Stella a lot to help me build a lot of the global files we want to add. Kind of ironic that its helping me help itself. It's almost like it's alive... Stella = Skynet? Hmmmm... They both begin with 'S' and have the same amount of letters!!

 

 

With these global files being separated, would the user have access to the folder still?

 

I like how their configuration file will be stored in the folder with their rom, because most of the time a homebrewer or hacker making their own config file will have that rom in a separate folder by itself anyway. It would be nice though if they could manually add it to the global file by physically putting it in there, if they choose.

[+stephena]

BTW, you can enter hex anywhere if you prefix it with a '%' symbol. The same is true for decimal if prefixed with '#'. It was also supposed to be possible to enter binary by prefixing with '\', but there's a bug that prevents typing the '\' symbol

 

I never knew, or thought of this. I'm so used to '%' being the default for binary, and '$' being the default for hex.

Actually, that was a typo. You use '$' for hex values, '#' for decimal, and '\' for binary. I wanted to use '%' for binary, but it's currently being used as the MOD operator. Maybe I'll remove the MOD operator, as I don't see much use for it.

 

Those that really want to make their own files will figure it out. I know I plan on using Stella a lot to help me build a lot of the global files we want to add. Kind of ironic that its helping me help itself. It's almost like it's alive... Stella = Skynet? Hmmmm... They both begin with 'S' and have the same amount of letters!!

In a way, it's the same way that having the debugger helps make the emulation more accurate. The debugger is for end users and developers, but it helps me just as much in debugging Stella itself.

 

With these global files being separated, would the user have access to the folder still?

 

I like how their configuration file will be stored in the folder with their rom, because most of the time a homebrewer or hacker making their own config file will have that rom in a separate folder by itself anyway. It would be nice though if they could manually add it to the global file by physically putting it in there, if they choose.

There will be a folder, which defaults to some location (still TODO), but can be modified by the user. When a ROM is loaded, Stella will first attempt to load a config file from the same directory as the ROM. If this fails, it will attempt to load from the global folder. The user is free to move their files into the global folder any time they like (outside of Stella, of course). But if another config file is ever placed in the ROM directory, it takes precedence again.

[Omegamatrix]

There will be a folder, which defaults to some location (still TODO), but can be modified by the user. When a ROM is loaded, Stella will first attempt to load a config file from the same directory as the ROM. If this fails, it will attempt to load from the global folder. The user is free to move their files into the global folder any time they like (outside of Stella, of course). But if another config file is ever placed in the ROM directory, it takes precedence again.

 

Perfect! I can start getting some configuration files ready for you too, but I'm not sure of the correct format needed. What I mean is I was going to use these:

 

[] for the bank

GFX for all gfx (hex and binary)

BDATA for block data

DATA

CODE

 

 

I'm still not sure what you are planning to do with the BIT, CMP, and NOP instructions yet though. I mean it'd be nice to have a singular directive that would look up the opcode and return it's name and length (-1).

 

So you could do BLAH (can't think of a label) like this:

 

 

BLAH F63E F63E

 

 

And it would return this in the code window:

 

.byte $0C ; NOP, skip 2 bytes | $0C

 

When it was a NOP absolute, or:

 

.byte $04 ; NOP, skip 1 byte | $04

 

When it was doing it for zero page, or:

 

.byte $24 ; BIT, skip 1 byte | $24

 

When it was doing a bit instruction.

 

 

 

The point I'm trying to make here is that the one directive handles all cases, and it expects you to pass it a byte range of 1 byte.

 

 

 

Also I'm terrible with names. Does anyone really want to call block data BDATA? Would a different name make more sense? I mean GFX, CODE, and DATA all make real good sense. Maybe BLOCK would make better sense. Maybe not... Any ideas??

[+stephena]

There will be a folder, which defaults to some location (still TODO), but can be modified by the user. When a ROM is loaded, Stella will first attempt to load a config file from the same directory as the ROM. If this fails, it will attempt to load from the global folder. The user is free to move their files into the global folder any time they like (outside of Stella, of course). But if another config file is ever placed in the ROM directory, it takes precedence again.

 

Perfect! I can start getting some configuration files ready for you too, but I'm not sure of the correct format needed. What I mean is I was going to use these:

 

[] for the bank

GFX for all gfx (hex and binary)

BDATA for block data

DATA

CODE

 

 

I'm still not sure what you are planning to do with the BIT, CMP, and NOP instructions yet though. I mean it'd be nice to have a singular directive that would look up the opcode and return it's name and length (-1).

 

So you could do BLAH (can't think of a label) like this:

 

 

BLAH F63E F63E

 

 

And it would return this in the code window:

 

.byte $0C ; NOP, skip 2 bytes | $0C

 

When it was a NOP absolute, or:

 

.byte $04 ; NOP, skip 1 byte | $04

 

When it was doing it for zero page, or:

 

.byte $24 ; BIT, skip 1 byte | $24

 

When it was doing a bit instruction.

 

 

 

The point I'm trying to make here is that the one directive handles all cases, and it expects you to pass it a byte range of 1 byte.

 

 

 

Also I'm terrible with names. Does anyone really want to call block data BDATA? Would a different name make more sense? I mean GFX, CODE, and DATA all make real good sense. Maybe BLOCK would make better sense. Maybe not... Any ideas??

You're right about '[]' being used for the bank. As for the new commands, I won't be getting to them yet, but I guess you could still add them (Stella will just ignore them for now). As for the naming and type, you'll have to discuss that with others, including Nukey. But as a I say, I won't be getting to this part for a while. The same is true for BLAH directive you mention. You're free to create config files for any (all?) ROMs you like, but they won't have full functionality yet.

 

Also, I won't be reachable for the rest of this weekend. I know that it's supposed to be a weekend of relaxation/camping/whatever, but I have a large pile a topsoil sitting in the backyard that needs to be spread

[Nukey Shay]

Current suggestions about the process:

 

Using Rom's V5 so that all contributions are consistant with each other.

 

Using a single "WIP" directory listing text file of that romset (with the unique long names) to minimize what needs to be downloaded by anyone wishing to contribute. Just download the text file and look for anything not completed within it.

 

Tagging completed directory entries with a bracketed number so that such files are easily seen by contributors. Preceding an entry with, for example, initials+# should keep contributions from accidentally using a reference number already taken for some other (unrelated) binary...and be short enough to fit Distella's 8:3 filename requirement.

ex:

[KH001]Rom's extremely long name of this game that Stella doesn't care about.bin

 

 

If an alternate title, alternate dump, or hack of that game uses the identical cfg info, one could just reference the number used...like this:

[KH001]Rom's extremely long name of an alternate game that Stella doesn't care about.bin

 

...or this...

 

[KH001]A gfx-only hack that Rom renamed using his monster filename method.bin

 

 

Actual .cfg info could either be in the same directory file...or just a seperate file all it's own which uses the same tag as the filename. For the latter, a small textfile KH001.cfg would contain the relevant info...holding the ORG/CODE/GFX/DATA info. Additional labels could be used within the file (for BLOCK/BDATA, for example)...or to add comments specific to that portion of the binary. Ex:

GFX F100 F13F // note: interlaced by 8 bytes

 

I'm not sure how far to carry the latter idea without the cfg potentially becoming as complex as a disassembly itself and/or otherwise being unusable for Stella / unreadable by a novice

 

 

Anyway...once cfg info is officially added to Stella, that tagged info can be eliminated from the WIP directory file...progressively shrinking it until no filenames remain (at which time, Skynet is active and Stella knows more than any programmer).

 

But what about the updates to the WIP directory text file? How should that be handled? Where do cfg files get sent?

[Omegamatrix]

Nukey and I (1 point for proper Englishy) have been playing Message tag all morning. Basically we're gearing up to start into this project and are just working out the details.

 

 

RomsV5 sounds like the reasonable base set to use. Per Nukeys suggestion I have made a directory list, editing it slightly in Excel. It looks like this:

 

 

[1737]4K    Video Pinball - Arcade Pinball (1981) (Atari, Bob Smith - Sears) (CX2648 - 49-75161) ~.bin
[1738]4K    Video Reflex (Foot Craz) (1983) (Exus) ~.bin
[1739]4K    Vogel Flieh (AKA Dolphin) (1983) (Quelle) (465.302  (PAL).bin
[1740]4K    Volleyball (AKA RealSports Volleyball) (1983) (Digitel).bin
[1741]4K    Volleyball (AKA RealSports Volleyball) (Dactari - Milmar).bin
[1742]4K    Volleyball (AKA RealSports Volleyball) (Double-Game Package) (1983) (Quelle) (781698) (PAL).bin
[1743]4K    Volleyball (AKA RealSports Volleyball) (Robby).bin
[1744]4K    Vom Himmel durch die Hoelle (AKA Parachute) (1983) (Quelle) (719.941 7) (PAL).bin
[1745]4K    Vulture Attack (AKA Condor Attack) (1982) (K-Tel Vision) (PAL).bin
[****]4K    Vulture Attack (AKA Condor Attack) (1982) (K-Tel Vision).bin
Dup of [0310]
[1747]4K    Wabbit (1982) (Apollo, Ban Tran) (AP-2010) ~.bin
[1748]4K    Wachroboter jagt Jupy (AKA Keystone Kapers) (1983) (Quelle) (715.853 5) (PAL).bin
[1749]4K    Walker (AKA Clown Down Town) (Suntek) (SS-032) (PAL) [a].bin
[1750]4K    Walker (AKA Clown Down Town) (Suntek) (SS-032) (PAL).bin
[1751]4K    Wall Ball (1983) (Avalon Hill, Duncan Scott) (5003002) ~.bin

 

 

So each line has a tag, and the duplicates are noted. Next the idea is to add the configuration files underneath each entry. Ideas right now involve adding comments to the configuration files (if possible). For example:

 

 

//Atari Point of Purchase Rom (Atari) NTSC
//MD5: 4EDB251F5F287C22EFC64B3A2D095504
[0]
ORG F000
CODE F000 F2CD
CODE F2CE F2D9 // code loaded into ram
CODE F2DA F3F1
BLOCK F3F2 F3FF
DATA F400 F43F
GFX F440 F47D
DATA F47E F4FB
BLOCK F4FC F4FF
DATA F500 F584
BLOCK F585 F5FF
GFX F600 F730
DATA F731 F760
GFX F761 F7D0
BLOCK F7D1 F7F9
DATA F7FA F7FF

 

The comment on the code loaded into ram lets anyone who pokes there nose into the configuration file know what's going on in this weird situation. I don't want to flood these files, just add the casual note here and there. The MD5 right in the configuration also seems like a good idea for matching.

[+stephena]

Nukey and I (1 point for proper Englishy) have been playing Message tag all morning. Basically we're gearing up to start into this project and are just working out the details.

 

RomsV5 sounds like the reasonable base set to use. Per Nukeys suggestion I have made a directory list, editing it slightly in Excel. It looks like this:

 

...

 

So each line has a tag, and the duplicates are noted. Next the idea is to add the configuration files underneath each entry. Ideas right now involve adding comments to the configuration files (if possible). For example:

 

 

//Atari Point of Purchase Rom (Atari) NTSC
//MD5: 4EDB251F5F287C22EFC64B3A2D095504
[0]
ORG F000
CODE F000 F2CD
CODE F2CE F2D9 // code loaded into ram
CODE F2DA F3F1
BLOCK F3F2 F3FF
DATA F400 F43F
GFX F440 F47D
DATA F47E F4FB
BLOCK F4FC F4FF
DATA F500 F584
BLOCK F585 F5FF
GFX F600 F730
DATA F731 F760
GFX F761 F7D0
BLOCK F7D1 F7F9
DATA F7FA F7FF

 

The comment on the code loaded into ram lets anyone who pokes there nose into the configuration file know what's going on in this weird situation. I don't want to flood these files, just add the casual note here and there. The MD5 right in the configuration also seems like a good idea for matching.

OK, just popping in because I'm waiting on an email from someone. I'll respond to your comments so far, before you get too far into this.

 

My plan was that the config files are named exactly as Stella would name the ROM in its internal database, but would end in .cfg instead. The reason being, it's then very easy to load a file. I don't need to load a directory and scan for for certain MD5. I can simply take the internal name, remove the ending and append .cfg, and then attempt to load it. When I said you guys should discuss the naming, I meant the naming of the new directives within the config file, not the name of the file itself. So I'll have to insist on the naming from Stella, since it makes things much easier on my end. Note that the names won't be too different from ROMs v5 collection, since I base the internal names in Stella on his naming scheme. For reference, you can get the latest stella.pro file here. If you prefer, there's also a C-style array containing the same info here. Each of these contain all the naming info you need.

 

As for the comments, it's fine to use '//'. I'll treat those lines as comments, where anything after them is ignored. However, adding comments to files created with the 'saveconfig' command will be more difficult. So for now, if you add comments and later edit and save the config file, they will be stripped. Probably better for comments to be added only to the final version of the file.