TI-99/4a disk-based CRPG

[PeteE]

Which LZW algorithm do you use? I'm looking into using LZW and Huffman coding to compress pattern tables. My goal is something very simple to decompress since I want the decompression code to take up as little space as possible. I've made a sprite compression scheme that takes advantage of horizontal/vertical symmetry and reflection/rotation, but it won't work as well on pattern tables.

[+adamantyr]

I use a custom one. I essentially created a library of common english letter combinations that are used by the bottom 31 characters. Characters 32-127 are stored as normal. 128-255 are used to denote a leading space character for the other characters. This gets me around 18-20% compression.

 

I tested using some LZW creator applications, but they all get way too complex too fast, attempting to encode the ENTIRE text block and use an entire word (16 to 32 bit) for encoding. Plus almost all of them strip out punctuation and are case-insensitive, which I NEED both of.

 

My only thought is if I could use the 128-255 range for more keyword replacement, I may get slightly better results, but I would need a means to analyze a large text block and cleanly identify the most optimal matches. I've decided for now to forge forward with my current system since it's better than nothing; I can always investigate better compression later.

 

Here's an assembly snip of my encoding/decoding algorithm:

 

 

* Size: 64 bytes
DECDAT DATA 0,4,8,12,16,20,24,28
       DATA 32,36,40,44,48,51,54,57
       DATA 60,63,66,69,72,75,78,81
       DATA 84,87,90,93,96,99,102,105
* Size: 108 bytes
DECTXT BYTE 3
       TEXT 'the'
       BYTE 3
       TEXT 'and'
       BYTE 3
       TEXT 'hat'
       BYTE 3
       TEXT 'ent'
       BYTE 3
       TEXT 'ion'
       BYTE 3
       TEXT 'for'
       BYTE 3
       TEXT 'tio'
       BYTE 3
       TEXT 'has'
       BYTE 3
       TEXT 'tis'
       BYTE 3
       TEXT 'are'
       BYTE 3
       TEXT 'you'
       BYTE 3
       TEXT 'can'
       BYTE 2
       TEXT 'th'
       BYTE 2
       TEXT 'he'
       BYTE 2
       TEXT 'an'
       BYTE 2
       TEXT 'in'
       BYTE 2
       TEXT 'er'
       BYTE 2
       TEXT 're'
       BYTE 2
       TEXT 'es'
       BYTE 2
       TEXT 'on'
       BYTE 2
       TEXT 'ti'
       BYTE 2
       TEXT 'at'
       BYTE 2
       TEXT 'of'
       BYTE 2
       TEXT 'to'
       BYTE 2
       TEXT 'ea'
       BYTE 2
       TEXT 'is'
       BYTE 2
       TEXT 'it'
       BYTE 2
       TEXT 'or'
       BYTE 2
       TEXT 'as'
       BYTE 2
       TEXT 'so'
       BYTE 2
       TEXT 'we'
       BYTE 2
       TEXT 'by'

* STRING is buffer for decoded text
* DECBUF is buffer for encoded text
DECODE MOV  R11,*R10+
       LI   R1,DECBUF
       LI   R2,STRING+1
       CLR  R3
DC1    CLR  R0
       MOVB *R1+,R0
       CB   R0,@B32
       JEQ  DCEND
       COCB R0,@B128
       JNE  DC2
       MOVB @B32,*R2+
       INC  R3
       ANDI R0,>7F00
DC2    CB   R0,@B32
       JLT  DC3
       MOVB R0,*R2+
       INC  R3
       JMP  DC1
DC3    MOV  R0,R4
       SRL  R4,7
       MOV  @DECDAT(R4),R5
       MOVB @DECTXT(R5),R4
       SRL  R4,8
       LI   R6,DECTXT(R5)+1
DC4    MOVB *R6+,*R2+
       INC  R3
       DEC  R4
       JNE  DC4
       JMP  DC1
DCEND  SLA  R3,8
       MOVB R3,@STRING
       B    @SUBRET

* STRING is buffer for decoded text
* WORK is buffer for encoded text
SPACE  DATA >2020
ENCODE MOV  R11,*R10+
       MOVB @STRING,R3
       SLA  R3,8
       LI   R0,STRING+1
       LI   R1,WORK
ENC1   MOVB *R0+,*R1+
       DEC  R3
       JNE  ENC1
       CLR  *R1
       CLR  R9
ENC2   LI   R2,WORK
       MOVB @DECDAT(R9),R0
       SRL  R0,8
       LI   R1,DECTXT
       A    R0,R1
       BL   @POS
       MOV  R4,R4
       JNE  ENC3
       INC  R9
       C    @W32,R9
       JLT  ENC2
       JMP  ENC4
ENC3   DEC  R4
       BL   @REPLC
       JMP  ENC2

* R4 = Position in string of insert point
* R9 = Character to insert
REPLC  MOV  R11,*R10+
       LI   R0,WORK
       LI   R1,128
RP1    CLR  *R0+
       DEC  R1
       JNE  RP1
       LI   R0,STRING+1
       LI   R1,WORK
       MOV  R4,R2
RP2    MOVB *R0+,*R1+
       DEC  R2
       JNE  RP2
       SWPB R9
       MOVB R9,*R1+
       SWPB R9
       
       B    @SUBRET

* R1 = Search string (length-byte at 0 position)
* R2 = Target string (No length byte, has character 0 at end)
* R4 = Return value (0 = not found, 1+ = position in string)
POS    MOV  R11,*R10+
       MOV  R1,R3
       MOV  R2,R4
       INC  R3
       INC  R4
       MOVB *R3,R5
       SRL  R5,8
POS1   CB   *R3,*R4+
       JEQ  POS2
       MOVB *R1,R5
       SRL  R5,8
       MOVB *R4,*R4
       JNE  POS1
       JMP  POSNO
POS2   INC  R3
       DEC  R5
       JNE  POS1
       MOVB *R1,R0
       SLA  R0,8
       S    R0,R4
       S    R2,R4
       JMP  POSEND
POSNO  CLR  R4
POSEND B    @SUBRET

 

 

[+adamantyr]

New blog entry up!

 

EDIT: Or not... Yahoo is being terrible with Wordpress lately, and it appears Wordpress is refusing to publish new posts (even though it SAYS it's published). The direct link is:

 

http://adamantyr.com/blog/2017/08/25/putting-it-together/

[+Vorticon]

Looking really good It's great to see that you seem to have little difficulty coming back to complex code and picking up where you left off even if it's been a while. I on the other hand really suck at this, and I have to literally relearn the entire code before picking it up again... This is particularly true for ALC.

[+adamantyr]

Looking really good It's great to see that you seem to have little difficulty coming back to complex code and picking up where you left off even if it's been a while. I on the other hand really suck at this, and I have to literally relearn the entire code before picking it up again... This is particularly true for ALC.

 

It helps that I comment my code pretty heavily, and the original structure has become pretty ingrained in my head...

 

That said, turning into a module-based structure helps because I do have to go over ALL the code. I haven't touched the travel portion of the code in years so I'm looking forward to going in and optimizing it.

[+adamantyr]

Did some analysis of my text encoder... it's actually pretty good, all things considered, with room for improvement!

 

I copied all my text for the first disk to a large text file and did some case-sensitive searches for my suffix replacement, and the results are interesting... The "value" listed is the length of the suffix multiplied by the number of occurrences.

 

Also note that the longer suffixes are done first, so "the" supercedes "he", which then loses count. This has been accounted for in the numbers.

Suffix	Count	Value
the	536	1608
in	635	1270
er	601	1202
you	324	972
and	319	957
es	450	900
re	439	878
ea	395	790
to	379	758
on	374	748
is	367	734
it	356	712
of	304	608
or	282	564
an	274	548
at	256	512
as	246	492
he	233	466
for	136	408
hat	134	402
ti	192	384
are	127	381
so	185	370
we	183	366
th	172	344
ent	104	312
can	76	228
ion	52	156
tio	39	117
has	39	117
by	36	72
tis	2	6

The last one, "tis", is definitely going to be replaced. I think what I'll do is replace the bottom five, requiring at least a value of 200.

 

[+adamantyr]

Okay, so I've written a new text encoder. It now uses 160 characters (0-31, 128-255) for dictionary key value replacement. I wrote up an encoder that takes in a dictionary list of values and produces a score sheet (how many of each was replaced) so I can actually see which ones work and which ones don't.

 

In order to determine the appropriate n-grams, I wrote up a utility to calculate ALL the n-grams in the given block of text that comprises the first disk. I think the word usage is general enough it should be re-usable for all the disks.

 

I also tested out using a bit-wise approach, with the most "common" letters being stored in 4-bit increments with a length nybble, while non-common characters are stored as a full 8-bit value. In practice, though, it didn't produce any better results.

[+adamantyr]

Just a heads up, you can see all my latest updates on this project on my blog!

 

http://www.adamantyr.com/blog

 

Solved my earlier problem with Yahoo by moving to AWS as well.

[+adamantyr]

Merry Christmas! Here is a video of the title screen music... I'm still making some adjustments here and there:

 

[Opry99er]

Hey, sweet!!!!

[+adamantyr]

Thanks! I actually just fixed it today so it has proper note decay... It sounds SO MUCH BETTER now.

 

I still hear the odd-ball off note here and there that I want to investigate. I know two of them I can do nothing about because the note is below the TI's range to create, but there's a few other anomalies I'd like to fix...

[Opry99er]

Sounds epic. Cant wait to dungeon crawl listening to that!

[+adamantyr]

Music is played only during the title screen.

 

Music and CRPGs can be done; Dungeons of Asgard does it quite well. I just decided I wanted to focus on sound for effects.

 

Also, I have no musical writing ability, and I readily acknowledge there is serious skill and talent required to make music that doesn't distract or get annoying.

[+mizapf]

Also, I have no musical writing ability, and I readily acknowledge there is serious skill and talent required to make music that doesn't distract or get annoying.

 

Burger Time. Nuff said.

[Opry99er]

Hen Pecked

[+adamantyr]

Oh yeah Burgertime...

 

All versions are maddening with the silly NEVER STOPS jingle music, but the TI is worse because Data East's software engineers apparently didn't understand they didn't HAVE to stop the music for every sound effect and start it over.

[+TheBF]

Music is played only during the title screen.

 

Music and CRPGs can be done; Dungeons of Asgard does it quite well. I just decided I wanted to focus on sound for effects.

 

Also, I have no musical writing ability, and I readily acknowledge there is serious skill and talent required to make music that doesn't distract or get annoying.

 

If you composed this piece then you have musical skills.

This is called polyphonic music best exemplified by J. S. Bach.

[+adamantyr]

 

If you composed this piece then you have musical skills.

This is called polyphonic music best exemplified by J. S. Bach.

 

I did not compose this piece. It's called "Caverns of the Heart", by Fred Nachbaur. He wrote it along with a number of other pieces for an unreleased CRPG in the early 2000's.

 

Sadly, he passed away around 2004, but I obtained permission from his daughter to use the song, so long as I give appropriate credit in both game and any manuals, which naturally I will do. (Although that was a LONG time ago... I may have to send her the completed piece and make sure she's all right with it's use again.)

[+adamantyr]

And here is the title screen with music playing...

 

[Opry99er]

 

RoA

 

 

How do I long for thee....

[InfernalKeith]

Just trying to imagine showing this to anyone in 1983 and telling them this was their TI 99/4A at work... heads would explode!

[Iwantgames:)]

Man this looks and sounds so good

[+adamantyr]

Okay, time for an update on this thread!

 

I'll be showing off the game and my development tools for it at Fest West this year. The engine is about 65% complete, combat is still in "rectification" from the old code base, and I won't have it anywhere near to done in time for Fest West. So I'm spending time on tuning and polishing the rest to make sure it looks good and plays well.

 

On a side note... If you try and visit my blog and it times out, it's due to a plug-in I installed called Jetpack to try and track metrics. The stupid thing seems to cause instability though, probably because it's trying to do something an AWS t2.micro can't really handle. I'm getting close to disabling it, for now, I've been just resetting the server when that happens because it's really useful to know how many visitors I'm getting.

[Opry99er]

Excited to see your progress, sir!!!

[+adamantyr]

Some mock-screenshots of the monster statistic screen. Doing stat screens is so memory consumptive, I think I'll have just enough space to get it done!