TIPI - TI-99/4A to Raspberry PI interface development

[+jedimatt42]

( @Asmusr the tipiboot.service just runs briefly during startup and then terminates ) 

 

I haven't gotten this running myself yet either... I've been fixing the rest of the software suite to work on python3 on my master branch. That seems ready for a release to all the TIPI users... I just want to do more testing this coming weekend. 

 

It looks like Pete sets the TIPI_WEBSOCKET environment variable to the path where the js99er build output is, so the websocket can host the files and all is delivered to the browser from one source.

 

I have @PeteE branch merged back upstream, with all of my other python3 fixes into https://github.com/jedimatt42/tipi/pull/124

 

Things build, but do not run on a 4A + PI setup... reading the code, I assumed they would as long as the websocket environment variable isn't set. The DSR code just hangs... the library will need to be instrumented and debugged.

 

I don't know, couldn't read the git docs on .gitmodules to understand how that sha1 library was meant to be pulled in... so I just cloned it in my setup.sh script. I assume there is some command from git to interact with that, but too tired, too much theory no examples when I google... bad googling skills tonight. 

 

I did find this: 

https://visualstudio.microsoft.com/visual-cpp-build-tools/

 

Looks like microsoft will provide a compiler and maybe we can get this built on windows....  This is going to be impossible for the average 4A user to use, unless it can be bundled as binaries... it needs to be: click... it's running... I'm not even interested in trying Qemu. 

 

 

[Asmusr]

7 hours ago, jedimatt42 said:

It looks like Pete sets the TIPI_WEBSOCKET environment variable to the path where the js99er build output is, so the websocket can host the files and all is delivered to the browser from one source.

Ahh, so that's what it's for. But I always run from https://js99er.net unless I'm changing the emulator.

[+jedimatt42]

@Asmusr running from js99er.net will be the ideal. Is there any browser security controls required to let the app use data from an alternative source such as this websocket?

[Asmusr]

1 hour ago, jedimatt42 said:

@Asmusr running from js99er.net will be the ideal. Is there any browser security controls required to let the app use data from an alternative source such as this websocket?

No the CORS policy doesn't seem to apply to websockets.

[PeteE]

I haven't tested it in QEMU yet.  My guess is that when the reset message kills the server, it doesn't restart the server socket in time for the websocket to connect.  The websocket should probably retry at some interval (5 sec?) if it can't connect initially.

[PeteE]

On 6/17/2020 at 11:59 PM, jedimatt42 said:

I don't know, couldn't read the git docs on .gitmodules to understand how that sha1 library was meant to be pulled in... so I just cloned it in my setup.sh script. I assume there is some command from git to interact with that, but too tired, too much theory no examples when I google... bad googling skills tonight.

I didn't really know how git submodules work either.  I discovered that you need to add the "--recursive" option to the "git clone" command, or run "git submodule update --init" if you've already cloned it without the "--recursive" option.  (from Stackoverflow)

[HOME AUTOMATION]

!!!YOU'RE THE MELVIN NOW!!!

[+jedimatt42]

1 minute ago, HOME AUTOMATION said:

!!!YOU'RE THE MELVIN NOW!!!

Can you explain that expression? I am unfamiliar with it.

[HOME AUTOMATION]

"65535"

Edited June 20, 2020 by HOME AUTOMATION

[Omega-TI]

20 minutes ago, jedimatt42 said:

Can you explain that expression? I am unfamiliar with it.

[PeteE]

Here's how to set up TIPI QEMU emulation for Windows:

1. Download and install QEMU
2. Create a folder "tipiqemu" and download the following items to it
3. Get tipi-sdimage-buster-1.43.zip (from Jedimatt42 download page)
4. Get kernel image and dtb file (from qemu-rpi-kernel)
5. Create a batch file "tipiqemu.bat" and paste the code into it

   "\Program Files\qemu\qemu-system-arm" ^
     -M versatilepb ^
     -cpu arm1176 ^
     -m 256 ^
     -hda sdimage.img ^
     -net nic ^
     -net user,hostfwd=tcp::9900-:9900,hostfwd=tcp::9901-:9901 ^
     -dtb versatile-pb-buster.dtb ^
     -kernel kernel-qemu-4.19.50-buster ^
     -append "root=/dev/sda2 rootfstype=ext4 rw"

6. Run the batch file and wait for the emulation to boot
7. Login as tipi/tipi and type the following commands

   cd tipi
   git pull
   git checkout merge-petee-master
   git submodule update --init
   sudo apt update
   nano services/tipi.sh
     (move down to the line containing "#export TIPI_WEBSOCK=..." 
      press delete to remove the "#" at the start of the line,
      press Ctrl-X and Enter to save the file)
   ./setup.sh           
     (type y when prompted)

    

8. Enable the TIPI emulation in js99er and make sure it's set to "ws://localhost:9901/tipi"
9. You should be able to access the normal TIPI web interface at "http://localhost:9900/"

 

Note: @Asmusr I have created another pull request with the retry operation for the websocket, with a 3-second timeout, that should allow enough time for restarting the tipi service.

Edited June 21, 2020 by PeteE

[+jedimatt42]

I should fix that setup.sh to not prompt for anything... I usually only use that when making an initial sd card image to distribute.  

I should also move the git submodule command and sudo apt update into that script as well. 

[Asmusr]

I think it takes even longer to set up (90 minutes and counting...) with the tipi image than it did with the generic image I used before. Isn't it possible to pre-install the dependencies on the image?

[Asmusr]

But finally it works!

[+jedimatt42]

Yes, a premade image is possible... once that image works for real 4A's I plan to release it all preset.. 

 

It didn't work for me last night, but I didn't try the patches to js99er yet... @Asmusr are they up on the website, or do I still need to pull Pete's PR and build locally? 

 

That process was a development upgrade process, that just hasn't been tested well enough yet, and well, the IF statements in the library weren't sufficient to work on both js99er and hardware. At least not as I tested it... I'll have to re-test, since I handled something slightly different than @PeteE instructions.  

 

I still would prefer to refactor things so that you get a python3 virtualenv in an .EXE running natively on windows. Reconfigured so some environment variables control where the tipi_disk folder is and fix whatever it takes to not be on a linux filesystem. The hard part of that will be re-integrating TidBit and xbas99 conversion that is currently delegated... but, that should be not very hard... just take time to work out how to install them correctly.

 

The QEMU approach, I think I want to preserve too, as that allows TIPI side development - emulation of the PI environment. 

[+jedimatt42]

@Asmusr I see version 8.0.1 deployed to js99er.net, is there any expectation of success or failure showing up in the LOG tab? 

 

I just get no interaction or feedback with the QEMU tipi setup, and nothing but a hang... 

[+jedimatt42]

@Asmusr I guess I don't understand the theory of emulation in js99er... 

 

When I pull up minimemory in js99er, none of the cru interface is implemented, so I don't know how 'it works' at all, or when the reset issue would have even been triggered. 

 

When inspecting in EASY BUG, ( is the TIPI at crubase 0x1000 or 0x1100 in this emulation? ) 

 

C1000  = 01

C1001  = 01

C1002  = 01

C1003  = 01

 

C1100  = 01

C1101  = 01

C1102  = 01

C1103  = 01

 

If the card is there, then whichever crubase is used, should have read zeros instead.  The rom for the card shouldn't be enabled unless bit xx01 is set... The reset should occur when bit xx02 is set. 

 

M4000  = 00 

 

That should have been AA  if the TIPI ROM is in the memory map... Since the TIPI ROM doesn't appear to be in the memory map, I don't understand how the basic subroutine in the ROM, 'CALL TIPI' is found and hanging... 

 

 

[+jedimatt42]

@Asmusr oh, I see, the cru just isn't readable... 

 

On tipi hardware, I made the crubits readable. 

[+jedimatt42]

@PeteE and @Asmusr it works... !  but I think I had to change the websocket back to Asmur's default  ws://localhost:9901/tipi ( why I changed it, was the result of a vision problem between screen and brain ) 

 

Best father's day gift ever!

[PeteE]

1 hour ago, jedimatt42 said:

@PeteE and @Asmusr it works... !  but I think I had to change the websocket back to Asmur's default  ws://localhost:9901/tipi

 

Best father's day gift ever!

js99er default is correct.  Sorry about that, it was getting late...

[+jedimatt42]

13 hours ago, PeteE said:

Here's how to set up TIPI QEMU emulation for Windows:

1. Download and install QEMU
2. Create a folder "tipiqemu" and download the following items to it
3. Get tipi-sdimage-buster-1.43.zip (from Jedimatt42 download page) 

 

Now use - tipi-js99er-beta1.64.zip    https://www.jedimatt42.com/downloads/tipi-js99er-beta1.64.zip  - this is just a snapshot of the qemu image after following @PeteE 's instructions. Then I disabled the tipiwatchdog.service which fails to start ( looks for GPIO pins ) 

 

Quote

1. Get kernel image and dtb file (from qemu-rpi-kernel)
2. Create a batch file "tipiqemu.bat" and paste the code into it

   
   "\Program Files\qemu\qemu-system-arm" ^
     -M versatilepb ^
     -cpu arm1176 ^
     -m 256 ^
     -hda sdimage.img ^
     -net nic ^
     -net user,hostfwd=tcp::9900-:9900,hostfwd=tcp::9901-:9901 ^
     -dtb versatile-pb-buster.dtb ^
     -kernel kernel-qemu-4.19.50-buster ^
     -append "root=/dev/sda2 rootfstype=ext4 rw"

3. Run the batch file and wait for the emulation to boot
4. Login as tipi/tipi and type the following commands    Updated js99er based image takes care of all this... 

commands for 4 are no longer needed. 

 

Quote

1. Enable the TIPI emulation in js99er and make sure it's set to "ws://localhost:9901/tipi"
2. You should be able to access the normal TIPI web interface at "http://localhost:9900/"

 

Note: @Asmusr I have created another pull request with the retry operation for the websocket, with a 3-second timeout, that should allow enough time for restarting the tipi service.    This change is live on the website.

 

Seems for best use, wait a moment after you go to the TI Title screen, as sometimes the service reset happens after you've begun something like Force Command's AUTOCMD or XB's LOAD  - I haven't really debugged that, it is an impression

(maybe that reset behavior should be made synchronous for js99er)

 

 

[+jedimatt42]

An interesting problem is the CIFS/Samba shares... so you are stuck with the TIPI web-ui or an SFTP client -- to use sftp, you'd need to alter in your launching windows bat file:

  -net user,hostfwd=tcp::9900-:9900,hostfwd=tcp::9901-:9901 ^

to

 

  -net user,hostfwd=tcp::9900-:9900,hostfwd=tcp::9901-:9901,hostfwd=tcp::10022-:22 ^

then you can sftp in as tipi user using localhost and port 10022 from the windows machine. 

 

theoretically. 

 

I also tripped over clicking the .bat from my windows gui, not starting in the same directory and having relative paths in the qemu command line... so add a CD command to your script to set the working directory, or create a windows shortcut that sets the working directory.

[+jedimatt42]

Ok, better than sftp... nfs... 

 

Using https://github.com/winnfsd/winnfsd

 

I made a directory on the windows machine, I called it tipi_disk

then cd into that directory from a cmd shell, and run:

 

WinNFSd.exe -id 999 995 . /tipi_disk

(The '.' can be a fully qualified path to the directory you want to share if you like... /tipi_disk is the name that will be exposed on the network. )

 

Then, in the QEMU console:

 

sudo mount -t nfs BYSS:/tipi_disk /home/tipi/tipi_disk
cp -R -a /home/tipi/tipi/setup/bin/* /home/tipi/tipi_disk/

( You can replace 'BYSS' with the name of your windows host )

 

Now you can just play with the files outside with whatever FIAD handling tools you want on windows. and use the TIPI native files handling easy... like the tidbit to PROGRAM handling or the XB to PROGRAM handling... or whatever you like to do with your TIPI filesystem.  Something similar could be done to replace where the PDF generation ends up... 

 

This is far more functional than I imagined... and all the tools seem to be redistribution friendly...  so instead of pursuing the windows native python approach, ( which would have cut features like PI.PIO and tidbit... ) I'll try and put together a .zip bundle that provides a simple, unzip and click - it's running. 

 

To get it mounted automatically when restarting the QEMU process, I add the mount command to my /home/tipi/tipi/services/tipiboot.sh   ( the sudo isn't needed in that script, it runs as root during startup ) 

[+9640News]

Matt,

 

Got a question.  Is there a way to flush the buffer for the socket read command without disrupting the connection?

 

With a serial RS232 connection to a BBS, most terminal software has a CTRL-C function that breaks the current routine on the server side.  Since it is single character based, there is not much data stored.  I can still send a CTRL-C with a TIPI socket connection, however the PI has already buffered or could have buffered a significant amount of data that is waiting to be read.

 

It would be nice to be able to empty the contents with a "FLUSH" socket command via keystroke on the TIPI client.

 

Alternatively, I could have the client read without processing the data to an empty large buffer location until things are empty but thought a FLUSH command to the Socket commands may be more appropriate.

 

Beery

[+jedimatt42]

3 hours ago, BeeryMiller said:

Matt,

 

Got a question.  Is there a way to flush the buffer for the socket read command without disrupting the connection?

 

With a serial RS232 connection to a BBS, most terminal software has a CTRL-C function that breaks the current routine on the server side.  Since it is single character based, there is not much data stored.  I can still send a CTRL-C with a TIPI socket connection, however the PI has already buffered or could have buffered a significant amount of data that is waiting to be read.

 

It would be nice to be able to empty the contents with a "FLUSH" socket command via keystroke on the TIPI client.

 

Alternatively, I could have the client read without processing the data to an empty large buffer location until things are empty but thought a FLUSH command to the Socket commands may be more appropriate.

 

Beery

I think you just have to read until the incoming read returns empty.. 

TCP doesn't handshake byte by byte like serial... a chunk of stuff was sent and transmitted already... it's there in the queue... to my knowledge it has to be drained before you can get at any future data.