gemini protocol browser: VIRGIL99

[+jedimatt42]

Bundled with ForceCommand 1.28, I've been working on a GEMINI protocol browser. 

 

Documentation on the protocol and purpose is over here: https://gemini.circumlunar.space/

 

I've got a tiny sample gemini server running at gemini://virgil.jedimatt42.com/

 

This VIRGIL99 browser requires TIPI, SAMS, F18A, a USB mouse connected to the PI, and ForceCommand.

This is incomplete. There are plenty of bugs. But I'm sharing in case anyone wants to provide feedback. 

 

 

The binary is in the ForceCommand release bundle ( in the included .DSK )

 

The command to run from ForceCommand is 'VIRGIL99' you can pass a gemini url to it as an argument. Or it will currently start with a built in 'about:' page. 

[+jedimatt42]

Here is a quick little video capture clip... 

 

 

[+jedimatt42]

VIRGIL99 v1.1 -

 

There is a new bin in the ForceCommand 1.28b zip : https://atariage.com/forums/applications/core/interface/file/attachment.php?id=886299

 

I've fixed a number of bugs: 

 

* I had misread spec, and assumed there was always some space between `=>` and the link... but that's optional. 

* added support for redirect responses from the server...  

* fixed error handling and status bar (at bottom) so it doesn't scroll the top menu bar off the screen.

* noticable, but small speedup of page loading. Much more speed is available...

* fix bug getting stuck in literal block mode if you interrupt the load while in a literal block. 

 

There are still some things where the content will for some reason corrupt the display. I will probably try to fix the loading speed first... 

Also I know long urls break things. 

[RXB]

Does what kind of PI you use affect speed of response?

I have a PI 3+ but most use a PI 1 so is there a speed difference as I know mine can address much more memory.

[+jedimatt42]

A PI Zero W or (now we have to call them 1W) has 1 core, and 512k of ram which is more than enough to keep up with the TI demands. I think you only experience a slowness in the upgrade process. There is a new PI Zero 2W released today that is 4 core and still 512k. I've ordered one and will post about that in the TIPI thread once I have vetted it.

 

The slowness I spoke of for VIRGIL99 is the result of my code maximizing overhead per byte transferred, instead of writing smarter code. This is time spent in the 4A, nothing to do with the PI.

 

 

[+OLD CS1]

2.4GHz IEEE 802.11b/g/n wireless LAN, Bluetooth 4.2, BLE

 

Still no 5GHz

[RXB]

3 hours ago, jedimatt42 said:

A PI Zero W or (now we have to call them 1W) has 1 core, and 512k of ram which is more than enough to keep up with the TI demands. I think you only experience a slowness in the upgrade process. There is a new PI Zero 2W released today that is 4 core and still 512k. I've ordered one and will post about that in the TIPI thread once I have vetted it.

 

The slowness I spoke of for VIRGIL99 is the result of my code maximizing overhead per byte transferred, instead of writing smarter code. This is time spent in the 4A, nothing to do with the PI.

 

 

Raspberry PI 3 + has 4 cores is what I use on my TIPI:

Specification

• Quad Core 1.2GHz Broadcom BCM2837 64bit CPU.
• 1GB RAM.
• BCM43438 wireless LAN and Bluetooth Low Energy (BLE) on board.
• 100 Base Ethernet.
• 40-pin extended GPIO.
• 4 USB 2 ports.
• 4 Pole stereo output and composite video port.
• Full size HDMI.

[+jedimatt42]

I don't know why we are talking about the size of your PI. And we shouldn't care.

 

VIRGIL99 implements the Gemini application layer protocol on the 4A, not the PI. 

 

We should keep this thread about VIRGIL99 and the Gemini protocol, please.

[wierd_w]

11 hours ago, OLD CS1 said:

 

 

 

Still no 5GHz

For goodness sake. It has a USB port.  Put a wifi dongle on if it bugs you that much.  CUDY makes a suitable low profile dongle, but the driver is out-of-tree, and has to be compiled.  Sources are on github. (Mucking with the Pi end of things is outside the scope of either the TIPI project this works with, or the Virgil99 project itself, but as long as you either disable the internal wifi chip through blacklisting the driver with modprobe.d, or change the interface name appropriately so it matches what the TIPI daemons expect, the rest of the TIPI package in the image should not give two shits about which physical interface is being used. Naturally, it would be entirely on you to get it set up and working.)

 

@jedimatt42

If the virgil protocol is implemented on the 4A side, does that mean you have implemented raw packet transfer to and from the PI?

 

Edited October 29, 2021 by wierd_w

[+Vorticon]

VIRGIL99 at the FC command line gives me an unknown command error. I have FC 1.28 installed.

EDIT: never mind. Found the executable under the FC/BIN folder

 

I have to say, it's pretty cool albeit quite slow still.

[+jedimatt42]

The PI takes care of the TCP layer, and the SSL layer over that and the TIPI stack offers that to the 4A as TCP sockets.

 

The 4A takes care of the 'gemini' application protocol layer. Just like my telnet app and my ftp app. Stewart's 99ML browser also implements HTTP on the 4A above the TCP socket provided by the TIPI stack.

 

You get a socket from the PI, and the 4A is responsible for all the bytes that are passed through..  

[+OLD CS1]

4 hours ago, wierd_w said:

Mucking with the Pi end of things is outside the scope [...] Naturally, it would be entirely on you to get it set up and working.

Precisely.

4 hours ago, wierd_w said:

For goodness sake.

What?  Am I not allowed to be disappointed about what I consider to be a missing feature?  I am not saying anyone has to provide it -- we have already too much of that around these places, anyway -- I am just expressing disappointment.

 

Trying not to shit up the thread, but since you mention it: I might get around to prioritizing my time to add Yet Another Thing to my hobby, to which I can only allocate a certain amount of time so I have to pick and choose: do I want to whimsically enjoy my hobby, or do I want to muck about with extra drivers and parts and compiling and dependencies and Broadcom firmwares and shit?  Being that I do the latter already on a day-to-day, my experience says it might be easy to do or it might be an absolute time-sucking pain in the ass.  Given that potential, I just skip it.

 

I have a 5GHz network.  I have reasons for not using 2.4GHz.  Some of the devices I want to use do not support that.  Not a big deal: I just skip them.  The same as a lot of things I use and do: my feature phone cannot do a lot of things anymore, so I just skip doing those things with it; my de-Googled Android phone cannot run certain apps, so I just skip those.

[RXB]

5 hours ago, jedimatt42 said:

I don't know why we are talking about the size of your PI. And we shouldn't care.

 

VIRGIL99 implements the Gemini application layer protocol on the 4A, not the PI. 

 

We should keep this thread about VIRGIL99 and the Gemini protocol, please.

Would it make more sense to use that PI memory for better code as you can load as many times as you want and only slowdown is the TI.

If optimized for size that would ensure slowdown of everything you do, but if you can load multiple sections with quick access that would speed up VIRGIL99

This would work like the SAMS does, the bigger storage is an advantage not for speed but for not having to compress programs to fit. Thus faster.

[+9640News]

Matt,

 

I think you have done a pretty amazing job with the display speed you have at present.  Between grabbing a line of buffer data, parsing it, mouse movement and mouse button control, then actually displaying the result, you are doing quite a bit.  I'm actually impressed of the display speed you have, and impressed you think you can improve upon the page loading.

 

I went to look at your github repository, and did not see it on there which is what I expected since it is a work in progress, as I was curious about some of your coding/parsing with the F18 displays.  I saw a lot of code for FC, but nothing jumped out that might be related to Gemini.

 

Are you reading in individual lines (small blocks )from the PI Buffer, displaying, then requesting another chunk from the PI buffer, or are you reading in a whole page into a SAMS buffer page(s) before attempting to display?  Just curious if you have time to answer.

Beery

[+arcadeshopper]

4 hours ago, 9640News said:

Matt,

 

I think you have done a pretty amazing job with the display speed you have at present.  Between grabbing a line of buffer data, parsing it, mouse movement and mouse button control, then actually displaying the result, you are doing quite a bit.  I'm actually impressed of the display speed you have, and impressed you think you can improve upon the page loading.

 

I went to look at your github repository, and did not see it on there which is what I expected since it is a work in progress, as I was curious about some of your coding/parsing with the F18 displays.  I saw a lot of code for FC, but nothing jumped out that might be related to Gemini.

 

Are you reading in individual lines (small blocks )from the PI Buffer, displaying, then requesting another chunk from the PI buffer, or are you reading in a whole page into a SAMS buffer page(s) before attempting to display?  Just curious if you have time to answer.

Beery

https://github.com/jedimatt42/fcmd/tree/main/example/gcc/virgil

[+jedimatt42]

Part of the magic is in the force command *_bufferedio.c  where I front the sockets with a 256 byte buffer, and allow you to take smaller chunks. If the 256 byte is drained, it will request more data from TIPI, and try and fill your application buffer.  I use that in the FTP program as well... there is also a 'readline' routine in there, that needs it's brain checked. It seems to work well enough for the FTP client, but FTP 'lines' are more strict than GEMINI lines.

 

So in VIRGIL99 I have another implementation of readline. Here is where I know it can go faster... right now the VIRGIL readline is using a single byte buffer to read from the bufferedio library in FC. This goes through all the bank-switching gymnastics to get me a byte.  If I get more bytes at a time from FCMD, it is definitely a lot faster, I've tested this... I just haven't gotten that to be correct yet... 

[+jedimatt42]

Oh, and I am reading lines into structures in SAMS. an Int and an 80 character array. Optimized for scrolling... the int tells me what gemini rendering mode to use, and then the array has the line item... as I read long lines from the server they have to be broken into 4A lines.  I use the sams pages mapped into F000 to hold 48 lines each page, and wrap that in a little code that given a line index, maps the correct page in, and returns the pointer to the line.  page.c holds that code. 

 

As I read lines into the page structure, if the line is in the range to display, I redraw the screen. I always redraw the top and bottom lines that are the VIRGIL controls, based on the state structure. Scrolling just changes the top line offset, and then redraws the screen. Any of the other interactions, set a new command, like reload previous URL for BACK, or simply STOP, or a link which is RELOAD a new url... Long operations notice that a new command is set, and abort what they are doing to unwind the stack back and then process the command. Long operations also process input in their loops if a VDP interrupt has passed, so you get to mouse around while it is loading a page. 

 

I use another page of SAMS at E000 for the url history. 

 

The program gets to live in A000 - DFFF..  Pretty small given what it does since so many routines are down in ForceCommand. 

 

The SAMS pages assigned are all dynamic... If you have SAMS, ForceCommand will have used some for itself, used some to load your parent programs such as FCMENUs, and used some more to load VIRGIL99.

 

It's a hoot having all the groovy bits in use. 

[+9640News]

13 hours ago, jedimatt42 said:

So in VIRGIL99 I have another implementation of readline. Here is where I know it can go faster... right now the VIRGIL readline is using a single byte buffer to read from the bufferedio library in FC. This goes through all the bank-switching gymnastics to get me a byte.  If I get more bytes at a time from FCMD, it is definitely a lot faster, I've tested this... I just haven't gotten that to be correct yet... 

Definitely aware of the speed issues I have seen on my end retrieving a byte versus a string.  Yeah, when you get that worked out, it will be substantially faster.

 

I read your other notes, and thanks for the overview.

[dgrissom]

On 10/24/2021 at 8:26 PM, jedimatt42 said:

Here is a quick little video capture clip... 

 

Jedimatt,

 

I noticed that your search box and bottom status lines have the correct black on green colors.

Mine always has a white font on green background.  The top line is the correct black on green.  The search, address, and return status all have that color combination.

It is almost unreadable.

F18 is 1.9.  Am I missing something?

 

DG

[+jedimatt42]

27 minutes ago, dgrissom said:

Jedimatt,

 

I noticed that your search box and bottom status lines have the correct black on green colors.

Mine always has a white font on green background.  The top line is the correct black on green.  The search, address, and return status all have that color combination.

It is almost unreadable.

F18 is 1.9.  Am I missing something?

 

DG

That's a bit weird... it should be taking over and setting all the colors correctly... was just about to drop a FC build, but let me revisit that issue in VIRGIL99... 

 

mysterious...

[dgrissom]

1 hour ago, jedimatt42 said:

That's a bit weird... it should be taking over and setting all the colors correctly... was just about to drop a FC build, but let me revisit that issue in VIRGIL99... 

 

mysterious...

Found the cause.

 

I "type" display an ANSI file in my AUTOCMD of a cartoon of my cat. (see attachments).

 

When I disable the ANSI image (CAT/TXT).  VIRGIL Displays correctly.  Could be a bug in Force Command or Virgil or just bug between my ears.

 

If I remove the image and my cat finds out, there will be hell to pay!

 

DG

 

AUTOCMD CAT.TXT

[+jedimatt42]

Ah, that's cool! So I'm failing to set my colors correctly  I should be able to reproduce it with this then

[+jedimatt42]

Ok, @dgrissom, give this a try:  https://atariage.com/forums/applications/core/interface/file/attachment.php?id=887112

 

 

[dgrissom]

4 hours ago, jedimatt42 said:

Ok, @dgrissom, give this a try:  https://atariage.com/forums/applications/core/interface/file/attachment.php?id=887112

 

 

My cat is very happy with this one! ?  Thanks Matt!

[dgrissom]

Virgil99 1.2 - Enjoying testing it!

Reporting a problem with:  values in |ADDRESS|  Address: after clicking on in page links.

If this is too early to give feedback.  Let me know?

 

I am testing Virgil99 along side of a Windows based Gemini Browser (GemiNaut).

 

I've found that the (some) links seem to be incomplete/garbled when parsed by Virgil99.  I after clicking get "Connection error".

 

Example:

The site:  "gemini://rainbow-100.com/"

 

Clicking on a link on the above page should give: "gemini://rainbow-100.com/computers/"

 

I get a connection error.  When I open up the Address box, the above text is there:  "0.com/index.gmiw_100.com/index.gmi" 

if I clear the line and type "//rainbow-100.com/computers/" the page displays correctly.

I don't know if this is a site quirk or?

 

The page source links (below). A leading "/' is not shown in the link. (Your Virgil page has it.):

=> rainbow.gmi The Rainbow 100 from Digital Equipment Corporation
=> next.gmi The Cube from NeXT
=> m102.gmi The 102 from Tandy Corporation
=> tt030.gmi The TT030 from Atari
=> as2000.gmi The Alphaserver 2000 4/233 from Digital Equipment Corporation

DG