Bitmap image conversions

[Opry99er]

Tursi,

I can only imagine what you've conjured up there.... I left this morning (6 AM) for a weekend in Washington DC playing at a club there... I don't have my laptop with me, so I won't be able to see what you have conjured up in your brilliance. . Maybe someone could do a short vid of it running--- post it here? Damn, I knew I'd rue the day I left my laptop at home!!!!!!!!!!!!

[sometimes99er]

Maybe someone could do a short vid of it running---

Something like this is something you have to experience !!! No previews please !

 

I stared in disbelief ... Was it or is it something from within Classic99 ? The file size could hold a picture ... I guess it does, and it's fucking unbelievable. It loads pretty fast (!?), so I can't explain what's happening. Very cool.

 

Edited February 18, 2011 by sometimes99er

[matthew180]

Damn, I knew I'd rue the day I left my laptop at home!!!!!!!!!!!!

 

Just go buy another one...

[+Vorticon]

OK where is the picture data? How did you hide it within the program file? I can just imagine XB splash screens using bitmap images. How come this has not been done before? I will test it on real hardware tonight and see if it still works. Very cool

Edited February 18, 2011 by Vorticon

[sometimes99er]

So, the XBTEST file is a TIFILES file format. Guess that in itself allows a file with different chunks and of different calibres (like "this is XB", this is a chunk to go at >BF00 etc.) ?

[Tursi]

No TIFILES doesn't allow anything like that. The TIFILES header is just a partial copy of the FDR from the disk.

 

The image data is appended after the program and loads with it.

 

I've written in detail how it works, and will publish the new version of Convert9918 tonight. When I do that I'll explain. But suffice to say there has always been a way to save data with an XB program... this is how Systex worked, and my own TMNT game used it for the digitized sound. I needed to rediscover how it worked, but I've done that and documented it. Just finishing off working out how to relocate an XB program so that I can also produce RLE versions (although I suspect small RLE versions won't work right if you modify them because of the backwards compatibility with BASIC...)

 

Glad to bring a little mystery into everyone lives though!

[sometimes99er]

That's wonderful.

 

I only started XB with assembler support last year, and gave up on it too - for anything serious. The object file was only getting close to 1K in size, and yet LOAD and overall experience was putting it down.

 

I do hope it works on the real deal, and preferable on emulators in general.

 

Even though this method obviously do not use object code (not EA3, but EA5 style), I'm still in shock over the gain in speed. I can't believe the disk system is actually that fast. You must be using magic.

 

[Tursi]

Dis/Var 254 at that size, actually.. but if you compare the load speed to any other 14k XB program, you should find it similar. However, the idle period after loading is shorter because there are so few lines! (However, if you load it an RES to resequence it, it will sit there for a long time. For some reason it still scans to the end of memory, even though it doesn't SEEM to affect the assembly code. I wouldn't trust it though.)

 

It should work on hardware, though I have not yet tried and nobody reported back. I have been putting off getting my hardware back online for a while, I suppose I should get to it. I even have a CF device now to supposedly make it easier, though I got it with the intention of writing a new DSR (only to find there's no docs on how the interface works, and it's buried in a CPLD. D'oh.)

 

Anyway, okay, didn't quite get done yet, but the notes! These will be included with the program when I publish it.

 

What does it do?

----------------

 

This is an Extended BASIC program with integrated assembly language and data that displays a bitmap screen, waits for the user to press Enter, then returns to Extended BASIC.

 

 

How do I use it?

----------------

 

The program file is generated from Convert9918 as a TIFILES standard file type. You can XMODEM this program to a real TI, or use Ti99Dir to copy it onto a disk image.

 

It loads and runs like any other Extended BASIC program.

 

 

How do I change it?

-------------------

 

This program messes with several of the system pointers in order to load and save larger than it really it. It seems to be okay to make small changes to it and save it again, but keep a backup and test often - sometimes it seems to get corrupted.

 

Only change code from line 200 onwards. For instance, you might want to change the background color in the CALL SCREEN to something more appropriate for your image. The CALL LINK("SHOW") does the actual display and waits for the user to press Enter before returning.

 

After a return, VDP RAM is a mess. You should always CALL CLEAR, and rather than add more code here, you should chain to your next program with RUN "DSK1.xxxx" (where 'xxxx' is your program).

 

The next program should always start with CALL CHARSET, otherwise all your character definitions will be corrupted.

 

 

How does it work??

------------------

 

This information was hard to come by, so it's worth documenting again. Extended BASIC builds a program starting at the top of memory. There are two pointers it uses during this, one points to the start of the line number table, the other to the end of the line number table.

 

When a new line is added, the line number table is shifted downwards in memory (this is why a large program starts to slow down when you add new lines). It puts the line building downwards from the start of the line number table pointer, then updates both pointers to the new table.

 

However, when Extended BASIC saves program code, it saves till the top of RAM no matter what the pointers say. Of course all this is only true for code in the 32k RAM.

 

This trick takes advantage of this behaviour by pushing the start and end points of the line number table lower in RAM. The program will then build downwards from that point, and SAVE and OLD will preserve all data right to the top automatically.

 

So first you have to start with NEW, because there must be no data in memory (otherwise updating the pointers without moving it will lose the program). The pointers are at >8330 and >8332 - just set them to just below the new top of memory -- although 1 byte lower is sufficient it's better to keep things aligned on even bytes, so I use two bytes lower. You can use CALL INIT/CALL LOAD to set these pointers, but in Classic99 I just use the debugger.

 

For this program, I set a value of >C9E4. This gives me 6k for the pattern table at >C9E6, 6k for the color table at >E1E6, 1k for saving VDP data at >F9E6, and 512 bytes for the program itself at >FDE6.

 

Next, enter the program. You can MERGE it, type it, or with Classic99 I just paste it in from Notepad.

 

To get the assembly into place, just CALL INIT::CALL LOAD("DSK1.ASM/O"). Because the assembly program is AORG'd, it loads into the correct place. Naturally you are responsible for making sure that space is free!

 

SAVE DSK1.PROGRAM -- and now you have an Extended BASIC program with space for almost 14k of data at the top, all loading in one fast loader with the assembly language in place.

 

Getting the image data in is trickier, this is why I put the code into Convert9918. You can use an external program to inject the data into the records once you find the correct points. To assist with this when I was testing it, I used the debugger to set the important addresses to a tag value I could search the resulting program file for (you could also use CALL LOAD). A file this large saves and loads as Dis/Var254 so even an Extended BASIC program could access the records.

 

It gets even trickier if you want to use RLE compression, which my loader supports. For this numerous offsets need to be recalculated. Convert9918 hacks the binary at the appropriate spots for this, or you could go through the above steps manually for smaller data sizes, and change the assembly program.

 

To break down the XB program:

 

100-130 are just warning comments. Normally I had no trouble making changes, but occasionally something got corrupted. Just use caution, keep backups and save often. Or better still, just chain to another program after displaying the picture instead of putting it all here.

 

140 does CALL INIT so the CALL LOADs will work.

 

150 Adds an entry to the name table called "SHOW " with an address of FDE6 - which is where our assembly is loaded. This LOAD assumes that no other assembly functions are loaded!

 

160 Changes the address of the bottom of the name table to >3FF8, so that CALL LINK can find our new entry. Again, it assumes that no other assembly functions are loaded.

 

170-190 are more paranoid warnings. Feel free to experiment though! Just don't be shocked if it crashes.

 

200 Sets the background color, as the assembly doesn't touch it. This may be useful for making the border color match your picture, so change if you like.

 

210 Displays the actual image, calling the assembly. The assembly code will wait for the user to press Enter before returning.

 

220 Clears the screen - VRAM is seriously messed up, so you should clear the screen right away to avoid showing too much corruption. I recommend that you also do a CALL CHARSET to reload the character set, but that can wait till the next program launches.

 

230 just notes that the next line should be replaced with a RUN command to launch your new program. Doing this is safer than continuing after messing around with the pointers -- but again, more power to you if it works!

 

240 prints a message as a test instead of running a new program, demonstrating that the interpreter is working.

 

To break down the assembly:

 

It first starts with some hard-coded equates to the buffer and the image, as well as to the XB version of KSCAN.

 

It then AORGs to the start address of FDE6, and the first instruction saves the return address to the interpreter to the first word of the buffer.

 

VWAD is a simple function to set the VDP address. By setting the bits appropriately, it can set for read or write mode, or change VDP registers. The first call in lines 21-22 set VDP Register 1 to >A0, which keeps 16k mode but blanks the screen.

 

Lines 25-26 set up and call a subroutine to copy specific blocks of VRAM to the buffer, backing it up. This is DATAST, and FROMVR is a function it uses to copy from VRAM to CPU RAM.

 

Lines 28-37 then set up the rest of the VDP registers for bitmap mode.

 

Lines 38-41 write >D0 to the first byte of the new sprite table, to turn off sprites.

 

Lines 43 to 55 set up a double-nested loop that writes 0-255 to the screen image table three times, thus setting it up for normal bitmap images.

 

Lines 61-70 set up and call the copy function for the pattern table, copying from TIAP to >0000. LOADDIR directly loads data to VRAM, while LOADRLE (not called in the source) unpacks an RLE table. Convert9918 patches this branch for RLE mode.

 

Likewise, lines 72 through 80 do the same for the color table, copying from TIAC to >2000.

 

Finally, likes 83-84 set VDP register 1 to >E2, which renables video and sets bitmap mode.

 

Lines 88 through 96 wait for the user to press Enter, enabling interrupts so that FCTN-= works. Note that if sprite automotion is in use, this will probably corrupt the picture since the sprite table is no longer where XB thinks it should be.

 

Lines 100-101 turn off the display again.

 

Lines 103-105 set up and call the copy function to copy data saved in the buffer back into VRAM.

 

Lines 107 to 199 change the VDP registers back to the defaults for a running XB program.

 

Lines 120-123 write >D0 to the XB sprite table, to turn off all sprites again (since the data in there is corrupt).

 

Lines 126-127 then restore the return address, and return to the XB interpreter.

 

Lines 130-136 is the routine to set a VDP address.

 

Line 138 is a bitmask used by the RLE unpack routine, which is in lines 140-184.

 

Lines 186-201 is the direct copy function.

 

Lines 203 to 220 is the data backup function. The data backed up was determined by observation and documentation, and there are three sets.

 

First, the data from >372 to >3ff is backed up. It seems like this may not be necessary, except for >3F0->3ff, which contains the graphic shape for the cursor and edge character. I haven't tested that theory enough to remove it.

 

Next the data from >820->B00 is backed up. Again, I'm not positive this is needed. Also, a larger program seems to use data beyond >B00, so if this IS needed that may cause a crash when XB resumes.

 

The last part is a little tricky - it backs up the Disk buffers up to >3800 (after that point we don't modify anything in VRAM). I wanted to support whatever configuration was in use, within reason, so we read the actual address for free RAM and subtract. If we get a negative number, then the value was higher than >3800 (this would be true for CALL FILES(2) or CALL FILES(1), or a system like Classic99 that does not use VRAM buffers, or even for cassette systems. Otherwise, it copies however much data is needed. Normally (for CALL FILES(3)) this is 40 bytes. If it is more than 146 bytes, though, the program will be overwritten and crash. The pointer is at >8370 if you need to check, If it is less than >376E then too much data is needed and you will need to rework it. This should only be true in unusual cases like multiple disk controllers or CALL FILES() greater than 3.

 

Back to the program, 224 to 232 is the FROMVR copy loop that copies from VRAM to CPU RAM.

 

Lines 234 to 242 is the TOVR copy loop that copies from CPU RAM back to VRAM.

 

Finally, lines 246 and 247 are used by Convert9918 to find the function pointers for LOADRLE and LOADDIR, so it can patch them. This was probably unneeded since much of the rest is hard coded, you definately don't need it for your own program.

[sometimes99er]

Very impressive.

 

Edit: I think a lot of future XB programs will be spiced up with this.

 

You've done a lot of work, and you're aware of it (the sprite thing), so I guess you're on to also backup of the sprite list and no. of moving sprites, set no. of moving sprites to zero, set the sprite list to point somewhere to >D0, and then, after the picture show, resetting it.

 

Edit: But then maybe the original character patterns are gone anyway ? Then at least no. of moving sprites to zero (no corruption).

 

Edited February 19, 2011 by sometimes99er

[Tursi]

The last big thing I was trying to solve before initial release was the relocation of the XB program so that I could save RLE images as well (smaller, so faster loading). I got that working fine in Classic99 and published the new version of Convert9918. However, it is NOT ready yet. I went ahead and tested with Win994A, MESS, and the real console, and it does not return successfully after you press enter (the picture DOES come up though!) So I need to troubleshoot that.

 

(edit) ah, just disregard most of my previous post asking for help. I worked out what is wrong on the other emus and real hardware, just a different block I need to backup/restore. Will fix that tommorrow. I think I may take the automotion suggestion too .

Edited February 19, 2011 by Tursi

[Opry99er]

I still haven't seen this run, but reading the breakdown.... Man, this is some serious work!!! Thank you so much, Tursi! I plan on utilizing this pretty heavily.

 

This really is an awesome community... The talent here... Wow

[Tursi]

I don't personally think it's super useful because of how much it adds to load time, but I'm happy to provide because it was a fun little hack. (Most people would usually rather get on with the game than wait to load a picture, hit a key, then wait to load the game).

 

Anyway, it now works as intended, and it's tested on Classic99, Win994a, MESS and real hardware with a CF7.

 

The following caveats come along with it:

 

-VDP RAM is trashed. That means you lose all your character definitions, and all sprites are turned off (I added a CALL DELSPRITE(ALL) to address Sometimes99er's suggestion), even the color table is messed. You can and should use CALL CHARSET in your next program to get the default characters and colors back, but you don't get lower-case with CALL CHARSET and so they, and all other characters, will be scrambled.

 

-The interpreter is not in perfect shape. It restores and runs, but some things are scrambled, especially if you go back to immediate mode. (For instance, I pressed Fctn-8 once to recall a line and it scrolled forever, and error messages corrupt the screen). I think chaining and continuing to run should be safe, but if you are still developing your code, always SAVE /before/ you run it, and always NEW /after/ you run it. Otherwise - your own risk of corrupting or losing your work.

 

Despite all the warnings, it is kind of fun! You could easily do a slideshow disk with this by chaining the programs to each other. Thanks for the challenge!

 

Fixed a bug that caused the Extended BASIC programs to crash after pressing Enter if a disk controller was installed. Also made the XB version delete all sprites before display (to turn off automotion).

 

http://harmlesslion.com/software/convert9918

[+InsaneMultitasker]

crash. The pointer is at >8370 if you need to check, If it is less than >376E then too much data is needed and you will need to rework it. This should only be true in unusual cases like multiple disk controllers or CALL FILES() greater than 3.

 

--

Beware the CF7/nanoPEB's use of VDPRAM @ 0x3ff8-3fff, which pushes the usual buffers down 8 bytes. 8370 is adjusted by the DSR at powerup and should be valid unless a program mucks around w/the value.

 

'tis time to try out this program of yours

[Tursi]

I don't assume the value of >8370, but there's a chance that the copy buffer ends up too small. 8 bytes is probably okay, and that paragraph isn't quite true anymore since all the pointers shifted around. Still, it worked on my CF7+, dunno whether that's different from the NanoPeb or not, but after a point I stopped worrying if it was right and only worried if it worked.

[Opry99er]

Excited to get home and try this out today... The theory you employ here could be used to add screen draw routines (similarly to systex).... I'm pretty excited about this--- though it is very different from the direction I THOUGHT you were going, it is super slick... And will make the "Classic99" version of the game easier to make. Thank you for doing this and documenting it.

[Tursi]

Not impossible to add more code, you could call load additional routines into low ram. Still, I do not recommend tying big XB programs in with the picture, I didn't test how well it tolerates longer programs.

[Opry99er]

It's quite interesting, really--- I wonder why the table wasn't made flexible in the first place... What you've accomplished seems like something that would be useful... Instead of putting a bitmap image at the top of memory, one could intermingle assembly within program lines--- it would be a similar premise, would it not? If I understand it correctly, this is how systex works.

 

I really wouldn't know the first thing about actually doing it, but this concept is interesting. I wonder why XB is so hard-wired... It would seem that having this kind of flexibility without wiping out all VDP and everything else would be possible, and certainly useful.... Is it possible that the need for GPL interpretation creates this inflexibility?

[matthew180]

I wonder why XB is so hard-wired...

 

Because the 99/4A does not have any RAM other than scratch pad and VRAM, therefore BASIC and XB have no choice but to use the video memory. Also, you are talking about a very resourced limited machine (as all home computer were), so you have to make a lot of trade offs and other compensations just to get everything to fix and have it work. This is why, for example, the MSX supports the VDP better from BASIC than TI BASIC. On the MSX they had real CPU RAM, so the VRAM was not tied up by the interpreter.

[Opry99er]

Tursi, I just had a chance to run this... I am completely amazed. This is faster than I'd ever imagined. I can't wipe this smile off my face. :) :)

 

http://www.youtube.com/watch?v=Bl5RJ_mfBWQ

[Tursi]

Hehe... glad you like it, Owen.

 

The Missing Link by Harry Wilhelm was a commercial package sold that added full bitmap graphics to Extended BASIC, including advanced sprite commands, line draw, boxes, fonts, windows, and the ability to load TI Artist files from disk. I wrote a TMNT game that used it for backgrounds, but when I contacted the author for permission to distribute it, he had to decline, largely because he was not sure whether he actually had the rights to give me (due to his contract with the distributor of the day).

 

So it's possible, but it's tricky. There are two problems. First is the one that Matthew mentions - that XB needs to run in an environment that has no CPU and ONLY VDP RAM available to it. So it needs to accept limitations. The other is that bitmap mode is hardware limited to the addresses it uses for the two largest tables - 6k at >0000 and 6k at >2000 - no choice (for full bitmap). That fragments your memory into smaller pieces and makes layout difficult. In fact the second table overlaps the disk buffers!

 

TML shuffles tables around and manages with some trickery a stable environment - it was no easy feat. Even with that, though, XB is still rather slow and bitmap mode just makes it even slower, as well as reducing how much stack RAM you have left (also used for strings).

 

Instead of putting a bitmap image at the top of memory, one could intermingle assembly within program lines--- it would be a similar premise, would it not? If I understand it correctly, this is how systex works.

 

Yes, this is how Systex works, and my program includes the assembly needed to load the image too. It's just raw CPU memory, so you can put anything there, it's actually kind of nice! For instance, you could load your character definitions and a short program for very fast initialization of character graphics, or you could include music programs with music. The XB loader is so much faster than the XB CALL LOAD that for anything large it is worth it.

[Opry99er]

Great stuff man. =) I've got some stuff I'd love to try out... problem is, I'm loading a TON of map data into low memory... I'm afraid that I might run out of room quickly. =)

 

For your loader... I'll be using this for sure... it's slick and fast. I'll be editing pictures to have image on top and text below it. Then drop the image into Convert9918 and BOOM! =)

 

It's very well done, man. Thank you for working this up. Very neat stuff.

[Opry99er]

here's a quick video of a linkup between 3 programs.... I tried to CALL SCREEN/CHARSET/DELSPRITE all after the first one, and it helped... but there's still some funky colors and VDP stuff going on between programs. =) It works very fast!!!

 

http://www.youtube.com/watch?v=hKD4ZTMq7HU

[Bones-69]

This really is so cool - those are some great images Owen. I wish this was around in 1984, I would have been sticking it so hard to my C64 mates.

[Opry99er]

You should see the file full of XB compatible images I've got now. . I spent about two hours yesterday finding images for Beryl, DLing them, converting them, and saving them. . Tursi is awesome!

[Tursi]

For anyone who wants to just randomly convert pictures in their image collection the EASY way, Convert9918 has a secret "pick a random picture" mode. I use this all the time myself.

 

1) Launch Convert9918 as per normal

2) Find an empty part of the dialog (for instance, above the 'Open' button, but below the black where the picture goes. Yes, the plain grey background.)

3) Double-click it with the left mouse button

 

If you did it right, the "Open" button becomes "Next". Click the "Next" button. A file dialog will open (just like for open), but it will be titled "Select any file in a folder for random load - filename will be ignored". Browse to the root of your normal PC picture collection, and select any file in the folder. (Note, if you don't have any actual files, for instance, the folder contains only subfolders, just make up a filename like 'x' and type it in. It gets ignored anyway.)

 

The console will show you that it is scanning the entire tree, and then it will randomly pick one picture and convert it. It will do this every time you click 'Next'.

 

To exit this mode, close Convert9918, next time you start it, it will be normal again.