TIPI - TI-99/4A to Raspberry PI interface development

[+jedimatt42]

32 minutes ago, Asmusr said:

There is no issue on the TI/emulator side. The problem in on the python side where I need to use something like https://websockets.readthedocs.io, which is event driven/asynchronous. It's not possible to poll a websocket, so it doesn't plug into TipiMessage as it is.

 

Oh, well you enable threading, and you have the event handler for the websocket dump the payload in a queue... and the tipiMessage code blocks on the queue... 

 

queue can have a length of 1, or basically be a mailbox. There are a few places where for convenience, one or both sides sends sends a couple messages sequentially.. that is done in the DSR to make it easier to send a value from this register, and then send a buffer from that hunk of ram, etc.. instead of forcing them to be aggregated. But they should always be serially handled. 

[PeteE]

I was working on a websockets interface for TIPI as a goal for emulation.  Let me clean up the code a bit and I'll share it.

[+9640News]

1 hour ago, jedimatt42 said:

 

Tursi's suggestion when TIPI prototype was first revealed was to provide something more natural... Like I do with XB source to PROGRAM conversion. 

 

If you try to LOAD opcode (or OPEN a DIS/FIX 128) a pc GIF or other image type, then the PI could trasform on the fly into a TIArtrist PROGRAM IMAGE file. 

 

I think you and InsaneMultitasker are the only ones running TIPIs on Geneves... so audience for Myart and GIF are pretty tiny... 

 

But, the same automatic mechanism could be modified to take target format hints... such as writing to PI.CONFIG a line like IMAGE_FORMAT=MYART, then try loading a JPG, and you get a Myart... 

 

That's the way I'd like to see that sort of interface work... convert in memory on load, leave native file in native file format on native filesystem. 

 

What do you want to do with these images ? How do you want to consume them? 

 

I mention MyArt, as that format is well documented and source code to decode the format for display on a 9938 is readily available.  What I did not know is if the F18A or MK2 would be able to handle a graphic display with it’s special abilities.  If the F18A or MK2 could not handle it in its special modes, then the number of 9938 users is probably pretty small to consider a GIF or MYART format.  Then, the TI_Artist format is more realistic.

 

I do not have a specific application program in mind, but it was something I thought of that someone may want to tackle as a project.  I do like your approach as that seems simpler.

 

It wasn’t something specific I was asking you to write as I was hoping my idea may be something that would be of interest to some of our graphic experts.

 

Beery

 

[Asmusr]

7 hours ago, PeteE said:

I was working on a websockets interface for TIPI as a goal for emulation.  Let me clean up the code a bit and I'll share it.

That would be helpful since I have not programmed in Python before. My concern is that if it differs too much from the master branch it would no longer be emulating the TIPI. 

 

Doesn't the current programming model mean that the RPi is running with high CPU load 24/7? I wouldn't want my emulator to do that.

 

[+jedimatt42]

1 hour ago, Asmusr said:

That would be helpful since I have not programmed in Python before. My concern is that if it differs too much from the master branch it would no longer be emulating the TIPI. 

 

Doesn't the current programming model mean that the RPi is running with high CPU load 24/7? I wouldn't want my emulator to do that.

 

 

There are sleeps in the polling if no data is ready at the first handshake, so that the average load on a pi-zero-w is around 10% and on a pi3 it is something like 2% - this was done specifically to avoid running the PI hot.

 

Regarding emulation accuracy... you just have to pick the level of accuracy you want... I have yet to see an emulator that reproduces the audible interference all my 4As produce when loading programs or feeding the VDP... 

I would recommend that we work together to reconcile changes into the release branch so that a configuration option controls the selection of the communication. TIPI is an unfinished, currently under active development project. So it would be a pretty bad idea to pretend we can keep 2 branches in sync... master is the development branch, people take updates from 'release' 

 

Having this available in emulation would make my life easier in many regards. So I'm happy to help. 

[+Vorticon]

Since the TIPI-PEB has not been available for a while, I'm going to go ahead and order my own PCB and populate it. I downloaded the GERBER zip file from the Github, so my question is does it contain all the needed files for the PCB manufacturer? Are there any other files I need to download. Also, any recommendations on a PCB house?

[+jedimatt42]

3 hours ago, Vorticon said:

Since the TIPI-PEB has not been available for a while, I'm going to go ahead and order my own PCB and populate it. I downloaded the GERBER zip file from the Github, so my question is does it contain all the needed files for the PCB manufacturer? Are there any other files I need to download. Also, any recommendations on a PCB house?

 

I actually have inventory to build about 10 of those. I just have been working on Force Command instead, and with lockdowns, my momentum working over at ArcadeShopper's workshop was interrupted... 

 

So at least 10  8 more are on the way. 

 

But, yes, for PCB printing the Gerber zip is all you need to upload. I don't have files for component assembly. I have used JLCPCB https://jlcpcb.com/

Edited June 6, 2020 by jedimatt42
Counted my inventory

[+arcadeshopper]

Also I have some being made I hope to have this month

Sent from my LM-G820 using Tapatalk

[PeteE]

On 6/3/2020 at 12:43 PM, Asmusr said:

Yesterday I looked into implementing TIPI support in JS99er. My idea was to use websockets as the interface between JS99er/TI and the python code, which would be slow, but I don't think that would matter for the emulation. I got stuck because the python code is implemented as a polling loop instead of being event driven. What would it take to reverse control so that TipiMessage is notified when a byte has been received instead of polling for it? For the normal code branch I guess it would mean moving the polling loop into libtipi. 

I pushed my websocket server to my tipi fork on github.  The websocket server is written in C, since that's more familiar to me than Python, and it's easier to implement a polling mode that works the same way as the gpio based calls.  It adds a new websocket mode in the libtipi module, enabled by uncommenting a TIPI_WEBSOCK environment variable in tipi.sh.

 

The websocket interface is at "ws://tipi.local:9901/tipi" and there are four registers RD, RC, TD, TC transferred in the text format "Rx=nnn" where n is the byte value in decimal.  You would map emulated writes to those byte registers at the top the TIPI DSR space to send that register to the websocket, and anything received from the websocket would update the byte registers to be seen by emulation.  I haven't fully tested the code, but I'm more than happy to debug any js99er changes you make.

Edited June 8, 2020 by PeteE

[Asmusr]

21 hours ago, PeteE said:

I pushed my websocket server to my tipi fork on github.  The websocket server is written in C, since that's more familiar to me than Python, and it's easier to implement a polling mode that works the same way as the gpio based calls.  It adds a new websocket mode in the libtipi module, enabled by uncommenting a TIPI_WEBSOCK environment variable in tipi.sh.

 

The websocket interface is at "ws://tipi.local:9901/tipi" and there are four registers RD, RC, TD, TC transferred in the text format "Rx=nnn" where n is the byte value in decimal.  You would map emulated writes to those byte registers at the top the TIPI DSR space to send that register to the websocket, and anything received from the websocket would update the byte registers to be seen by emulation.  I haven't fully tested the code, but I'm more than happy to debug any js99er changes you make.

Excellent, I will try to add some emulation tonight. Where do you imagine the Python code would run from? An actual RPi is one option, of course, but that's probably asking too much of the regular JS99er user. I imagined putting it on a server, but that would require some kind of session/user identification. Is there anything in the code binding it to the RPi or Linux?

Edited June 8, 2020 by Asmusr

[+jedimatt42]

1st: It is all written on the premise that the underlying filesystem is linux.

2nd: Package dependencies are raspbian based. smbd is a part of the TIPI experience. There are PI specific packages that would need to be made optional... 

3rd: I don't know how to use windows well enough to make python work on it, or compile native code on it or do anything other than 'Steam' on it. 

 

If people aren't going to just get a PI, and run it on the PI, then this should not be merged into the release branch, but instead be maintained as a downstream branch/fork. I can't commit to testing changes on multiple platforms. There are major changes that need to come relatively soon, like porting to python3, and upgrading from "raspbian" to "Raspberry PI OS".  The core service for the filesystem and networking also has the stupid OLED support that I intend to remove first which will remove a very PI specific piece.  If you go the fork route, we don't have to flood the code with conditionals, you just have a separate system that works in this new way, and upstream changes are merged out to the downstream system as time allows. 

 

Who is the regular JS99er user?  I don't use emulators for anything other than as a debugger, so I'm completely unqualified to characterize the target audience...

 

Are you trying to provide the JS99er user the TIPI filesystem? 

TIPI networking? 

other TIPI extensions? or even the TIPI extensibility model? 

PDF generation? 

 

Does the typical JS99er user even matter? Or are you trying to provide yourself a debugger for developing toward some subset of TIPI features? 

[PeteE]

22 minutes ago, Asmusr said:

Excellent, I will try to add some emulation tonight. Where do you imagine the Python code would run from? An actual RPi is one option, of course, but that's probably asking too much of the regular JS99er user. I imagined putting it on a server, but that would require some kind of session/user identification. Is there anything in the code binding it to the RPi or Linux?

I'm running it from an actual RPi.  TIPI is inherently a single-user system, expecting to be connected to a single TI computer through the sideport.  For emulation, we would switch the wire gpio interface to the websockets interface (through the presence of the TIPI_WEBSOCK environment variable) connected to a JS99er page served from the RPi.  When in websocket mode, it can no longer use the gpio mode to the TI, and vice versa.

[Asmusr]

10 hours ago, PeteE said:

I'm running it from an actual RPi.  TIPI is inherently a single-user system, expecting to be connected to a single TI computer through the sideport.  For emulation, we would switch the wire gpio interface to the websockets interface (through the presence of the TIPI_WEBSOCK environment variable) connected to a JS99er page served from the RPi.  When in websocket mode, it can no longer use the gpio mode to the TI, and vice versa.

I have pushed a branch to https://github.com/Rasmus-M/js99er-angular/tree/tipi

I can't test it because I only use Windows. If the Python code can only run on a RPi I'm not likely to go any further with this. 

[Edit: I pushed a change because I think I got the addresses wrong.]

Edited June 9, 2020 by Asmusr

[+jedimatt42]

6 hours ago, Asmusr said:

I have pushed a branch to https://github.com/Rasmus-M/js99er-angular/tree/tipi

I can't test it because I only use Windows. If the Python code can only run on a RPi I'm not likely to go any further with this. 

I'm sure we can prune the TIPI junk that is OS dependent, and just get the TipiService.py to run on windows. Maybe we can use something like pyinstaller.org to create an easy to consume executable... we don't need the web-ui, the monitor service, the super-service... the upgrade mechanism, can be tossed in favor of new binary distribution. And the OLED crap should just quietly fail if it isn't supported. So just focus on the 1 process. 

 

tinames.py probably needs to be changed to expect files in the user's home-directory under tipi_disk instead of the hard-coded linux path I have in there. 

 

That should get file and network support working... I have no idea how mouse would work... cause that relies on reading the unix /dev/mice character device file directly.  But that is just a novelty... 

 

There are some nice features that won't work without re-engineering... PI.HTTP PI.HTTPS file access, depends on shelling out to the OS, cause that was easier than the earlier native python curl library I was using... The TiDBit automatic conversion on LOAD, and file.xb automatic conversion on LOAD rely on invoking matthew180's PHP code and Ralph B. xbas99 as command processes... It'll take more work to get those installed on windows... 

 

[PeteE]

It might also be possible to virtualize the TIPI server using QEMU: https://www.instructables.com/id/Raspberry-Pi-Emulator-for-Windows-10/

[Asmusr]

23 hours ago, PeteE said:

It might also be possible to virtualize the TIPI server using QEMU: https://www.instructables.com/id/Raspberry-Pi-Emulator-for-Windows-10/

I got as far as this [using your fork]:

Edited June 10, 2020 by Asmusr

[+jedimatt42]

1 hour ago, Asmusr said:

I got as far as this [using your fork]:

 

@PeteE has done some more work today, I believe he had a patched up version of things running under Qemu... and then broke free of the Raspbian barrier.  He is out ahead of all of us, so I'd let him role with it... and just be patient. 

 

I've started looking at pyinstaller, and cleaning up some of the unnecessary PI only stuff. He has adapted things to Python 3... and running loose on a linux desktop. 

 

We should be able to get this to be an .exe on windows, the user just runs without an ARM emulator. 

 

---

One js99er note, reviewing the change set, I noticed a lack of any code looking like cru bit handling? The second TIPI crubit will be important to signal the python software to reset when the powerup code runs... the others can be ignored unless I eventually utilized the bank-switched ROM. 

 

[+jedimatt42]

There are a couple programs out there that I am hereby declaring, not TIPI protocol compatible. They may work on current incarnations of TIPI, but I will not promise they work on TIPI emulation, or future TIPI hardware designs.

 

I put the andresses of the TIPI sendmsg, vsendmsg, recvmsg, and vrecmsg routines at well known locations in the ROM for a reason. So those routines can be used, and on the python side, the TipiMessage.py layer is the counter-part. I want to be able to change the implementations of those routines without breaking software written to utilize TIPI functionality. This created an abstraction, I call the TIPI Messaging API

 

To date, if I change those protocols, the latest piece of software from @InsaneMultitasker ( that I'm aware of ) will break. And the libtipi for MDOS by @BeeryMiller will break.  I am ok with declaring those programs 'across the line' and not compatible with future versions of TIPI. @InsaneMultitasker 's reason for doing this can probably be fixed elsewhere and a solution incorporated into the official messaging protocol. @BeeryMiller 's reason for doing it is more about physical compatibility issues with the Geneve. Given those routines work from ROM in ROMPAGE mode, I believe this isn't a technical limitation, just a gap in understanding exactly how to setup in MDOS. 

 

I always intended to be able to go back in and replace the serial bus hardware protocol with a parallel bus design, to improve performance. (For instance, there are 8 signal lines between the PI and TIPI board. Each are GPIO, and software reprogrammable. A 4 bit parallel bus could be made on existing hardware. A direction signal, a clock signal, the hard reset, 4 data lines, and a parity signal.) But on 4A hardware, performance has been good enough. However with the emulation support on the horizon, my software experience tells me we are going to have to take advantage of the provided abstraction of the TIPI Messaging API.

 

So I am reserving the right to state that programs dipping below that protocol may get left behind. They are not TIPI Messaging API compatible. It is a TI... I've left behind any program that reads sector 0 to figure out where a file is... So, take it or leave it... As time permits, I am happy to help fix the issues that have driven a couple programs to go below this layer.

 

I've documented the low level wire protocol: https://github.com/jedimatt42/tipi/wiki/TIPI-Protocol

 

On that page, I have updated the assertion that the TIPI Messaging API is the compatibility layer. 

[Asmusr]

9 hours ago, jedimatt42 said:

One js99er note, reviewing the change set, I noticed a lack of any code looking like cru bit handling? The second TIPI crubit will be important to signal the python software to reset when the powerup code runs... the others can be ignored unless I eventually utilized the bank-switched ROM. 

Are the CRU bits only for the DSR to read, so I just need to store them?

[+InsaneMultitasker]

27 minutes ago, jedimatt42 said:

To date, if I change those protocols, the latest piece of software from @InsaneMultitasker ( that I'm aware of ) will break. And the libtipi for MDOS by @BeeryMiller will break.  I am ok with declaring those programs 'across the line' and not compatible with future versions of TIPI. @InsaneMultitasker 's reason for doing this can probably be fixed elsewhere and a solution incorporated into the official messaging protocol

If I understand correctly.... so long as the well-known addresses are used and called to access the TIPI messaging protocol, compatibility is not an issue.  I am using the "well-known" addresses for the TI Mode emulation so no concerns there. If the program you refer to is Mass Transfer, I consider that a stepping stone to learning more about the TIPI and not a program of consequence.  As for the Geneve OS, I assume that if the messaging protocol requirements changes, the EPROM (DSR) would need to be updated;  so long as the code is available I can pivot as needed.  I have a few ideas about adding 'well known' Geneve support to the EPROM that I will run by you if/when I make it far enough along to be of any value.  I have a lot more research to complete within the OS beforehand and with recent work resurgence, little time to do so.

 

[+jedimatt42]

20 minutes ago, InsaneMultitasker said:

If I understand correctly.... so long as the well-known addresses are used and called to access the TIPI messaging protocol, compatibility is not an issue.  I am using the "well-known" addresses for the TI Mode emulation so no concerns there. If the program you refer to is Mass Transfer, I consider that a stepping stone to learning more about the TIPI and not a program of consequence.  As for the Geneve OS, I assume that if the messaging protocol requirements changes, the EPROM (DSR) would need to be updated;  so long as the code is available I can pivot as needed.  I have a few ideas about adding 'well known' Geneve support to the EPROM that I will run by you if/when I make it far enough along to be of any value.  I have a lot more research to complete within the OS beforehand and with recent work resurgence, little time to do so.

 

 

Perfect. 

 

[+jedimatt42]

59 minutes ago, Asmusr said:

Are the CRU bits only for the DSR to read, so I just need to store them?

 

crubit 1 - enables the rom, and LED like most PEB cards. 

crubit 2 - needs to send a message over the websocket, like RESET or something the other end can parse. This is literally a wire from the TI to the RPi, that triggers an interupt on the PI GPIO and causes a second process to kill -9 the tipi.service, so anytime a user changes a cartridge or hits their hardware reset button, the powerup routine in the ROM manipulates this crubit, and the hardware interrupt on the PI is triggered. We'll have to do something different in emulation... but the software interface to it, is crubit 2. 

crubit 3 & 4 - select the high address bits of the DSR ROM. Not used presently... The ROM turned out smaller than anticipated. 

[Asmusr]

52 minutes ago, jedimatt42 said:

 

crubit 1 - enables the rom, and LED like most PEB cards. 

crubit 2 - needs to send a message over the websocket, like RESET or something the other end can parse. This is literally a wire from the TI to the RPi, that triggers an interupt on the PI GPIO and causes a second process to kill -9 the tipi.service, so anytime a user changes a cartridge or hits their hardware reset button, the powerup routine in the ROM manipulates this crubit, and the hardware interrupt on the PI is triggered. We'll have to do something different in emulation... but the software interface to it, is crubit 2. 

crubit 3 & 4 - select the high address bits of the DSR ROM. Not used presently... The ROM turned out smaller than anticipated. 

I have committed some code that sends "RESET" if cru bit 2 is set. I have also enabled setting the websocket URI in the UI.

[+9640News]

1 hour ago, jedimatt42 said:

 

To date, if I change those protocols, the latest piece of software from @InsaneMultitasker ( that I'm aware of ) will break. And the libtipi for MDOS by @BeeryMiller will break.  I am ok with declaring those programs 'across the line' and not compatible with future versions of TIPI. @InsaneMultitasker 's reason for doing this can probably be fixed elsewhere and a solution incorporated into the official messaging protocol. @BeeryMiller 's reason for doing it is more about physical compatibility issues with the Geneve. Given those routines 

At the time I had issues, I was programming with a GenMOD Geneve.  The GenMOD Geneve is in another system.  I will go back and review the sndmsg and rcvmsg with a stock Geneve to see if there is a difference resorting back to using the routines within the DSR itself.  Right now, I just do not recall what the specific issue was as that was almost 2 years ago.  If it works, then I will test on the GenMod system.  

 

As far as MXT, I just do not recall how I implemented the use of sndmsg and rcvmsg as that version of MXT could only run on a 4A anyways and was only tested on a 4A.  I presume Tim modified the TIPI source I had, but I haven't seen the source to know what he may have did.

 

I do know AfterHours does use the copied sndmsg/rcvmsg routines from the DSR in its code.  AfterHours is runnable on both Geneve and TI-99/4A systems.

 

I will first look at my TIPI driver source code to see if it can be changed.

 

A question though from where you are talking about serial and parallel designs of the TIPI.  Is this going to result in a TIPI V2 board, or is it just coding changes to make use of the existing hardware in a different approach?

 

Please note I am very interested in the emulation support from the viewpoint it would sometimes makes debugging easier to see what is happening.

 

Beery

 

 

[+jedimatt42]

I'm not promising anything, I have a lot of projects on my plate.

 

Technically, a much faster protocol could be implemented on existing hardware. And I would like to eventually explore that.  I'm from the early 70s, won't be able to retire for a while, but that doesn't stop me from queuing up projects... LOL... and my priorities shift constantly. Theoretically, I could turn 18 io clocks from the PI into 4, per transferred byte of data. 

 

The priority here is keeping things open for alternative implementations.