fujinet #FujiNet - a WIP SIO Network Adapter for the Atari 8-bit

[damosan]

I'm going to try to build an asm version of the CIO routines this Sat/Sun using the relocator.  I sent a pull request yesterday that makes the relocator a bit smarter in handling data.  This week I'm going to add support for an init function that will be overwritten after it runs to conserve memory.  The intent is to pop into Basic with everything required to send/rec on the N: device using as little RAM as possible.

 

When i get my grubby hands on a device things will roll a bit quicker - or someone can build the code and try it out and watch it all implode. 

[tschak909]

1 minute ago, damosan said:

I'm going to try to build an asm version of the CIO routines this Sat/Sun using the relocator.  I sent a pull request yesterday that makes the relocator a bit smarter in handling data.  This week I'm going to add support for an init function that will be overwritten after it runs to conserve memory.  The intent is to pop into Basic with everything required to send/rec on the N: device using as little RAM as possible.

 

When i get my grubby hands on a device things will roll a bit quicker - or someone can build the code and try it out and watch it all implode. 

I'm glad things are coming together... now if I could just get somebody to look at esp32/tests/atxtest and see what i did wrong? @TheMontezuma?

 

-Thom

[mozzwald]

15 minutes ago, damosan said:

When i get my grubby hands on a device things will roll a bit quicker - or someone can build the code and try it out and watch it all implode. 

Hopefully finish up the FujiNet build tonight and ship tomorrow. Shouldn't be too long before it's in your hands

 

[Quiver]

18 minutes ago, mozzwald said:

Hopefully finish up the FujiNet build tonight and ship tomorrow. Shouldn't be too long before it's in your hands

 

Any chance that you could point us to where we could get the STL files for those SIO connectors?

[mozzwald]

1 hour ago, Quiver said:

Any chance that you could point us to where we could get the STL files for those SIO connectors?

I was not planning to release the design files until we have a finished (or close to) device. While the current connectors fit and work, they need adjustments. For example, the plug is actually too small but the pins are holding it in place which is not ideal and the receptacle is too short. The next revision may not even use this design.

 

That is, unless there is someone with 3D design skills that wants to work on a better design

Edited February 4, 2020 by mozzwald
clarification

[Quiver]

Okay, cool. Would rather have a properly fitting connector. It's just very hard to find a good SIO receptacle these days, and seemingly impossible to find a 3d printable one.

[mozzwald]

2 minutes ago, Quiver said:

Okay, cool. Would rather have a properly fitting connector. It's just very hard to find a good SIO receptacle these days, and seemingly impossible to find a 3d printable one.

I have tinkered with making a replica SIO receptacle. The idea uses the same pins which are soldered thru hole to a tiny pcb with another set of pins offset 90 degrees to mate with the atari pcb. It's still very early for this design though. I also have some different style pins coming that may or may not work better for the replica and/or FujiNet.

[TheMontezuma]

3 hours ago, tschak909 said:

I'm glad things are coming together... now if I could just get somebody to look at esp32/tests/atxtest and see what i did wrong? @TheMontezuma?

 

-Thom

Sorry, but I know nothing about ATX and copy protection... Maybe @Farb could help?

He is an expert in those things and also has experience in Arduino programming (afaik).

[tschak909]

on a mundane, but still very critical aspect of the TCP firmware code,  There has to be a bulletproof way to parse out an N: device spec.

 

Tenatively, the current N: device spec is:

Nu:<PROTOCOL>:<PATH>:<PORT>

Where:

• u = the desired unit, 1 - 8, or not specified, which defaults to 1.
• PROTOCOL, the desired protocol, examples here are TCP, UDP, HTTP, HTTPS, SSH, FTP, etc.
• PATH, the majority of the path, varies from protocol to protocol, for TCP, a hostname, for HTTP, a URL, etc.
• PORT, the port number

 A variant of this, is for creating listening sockets, where:

Nu:<PROTOCOL>:<PORT>

• u = the desired unit, 1 - 8, or not specified, which defaults to 1.
• PROTOCOL, the desired protocol, examples here are TCP, UDP.
• PORT - the port # to listen to.

Examples of some devicespecs passed to OPEN:

• N:TCP:192.168.1.8:2000 (connect to 192.168.1.8 port 2000)
• N:HTTP://WWW.GOOGLE.COM:80 (connect to WWW.GOOGLE.COM port 80)
• N:HTTPS://WWW.IRATA.ONLINE:443 (same for HTTPS)
• N:FTP:FTP.PIGWA.NET:21 (FTP server)
• N:TCP:2000 (Listen for TCP connections on port 2000)
• N:UDP:4000 (Listen for UDP packets on port 4000)

And so on.

 

To parse this, I've had to overhaul the code in the ESP firmware to cleanly strtok() the input without crashing, and handling the possible valid cases, marking the rest as invalid, resulting in the following code:

/**
 * Parse a devicespec
 *
 * Params:
 *  char* devicespec - input  - device spec (N:TCP:FOO.COM:2000)
 *
 * Returns:
 *  bool  Valid device spec or not.
 *  char* devicename device name (e.g. 'N' 'N1')
 *  char* protocol   protocol (e.g. TCP)
 *  char* path       path component (FOO.COM)
 *  unsigned short port the parsed port (2000)
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>

bool devicespec_parse(char* devicespec,
                      char* devicename,
                      char* protocol,
                      char* path,
                      unsigned short* port)
{
  char* p=strtok(devicespec,":");
  int i=0;

  while (p!=NULL) // loop through until all tokens processed.
    {

      if (i==0) // check for valid devicename
        {
          if (p[0]=='N')
            strcpy(devicename,p);
        }
      else if (i==1) // protocol
        {
          strcpy(protocol,p);
        }
      else if (i==2) // path or port
        {
          if (atoi(p)==0)
            {
              strcpy(path,p);
              *port=0;
            }
          else
            *port=atoi(p);
        }
      else if (i==3) // port number
        {
          *port=atoi(p);
        }

      p=strtok(NULL, ":");
      i++;
    }

  // Parameter validation
  if ((i==1) || (i==4))
    return true;
  else if (i==3)
    {
      if ((path[0]=='\0') && (*port!=0))
        return true;
      else
        return false;
    }
  else
    return false;
}

int main(int argc, char* argv[])
{
  char devicename[3];
  char protocol[8];
  char path[256];
  unsigned short port;

  memset(devicename,0,sizeof(devicename));
  memset(protocol,0,sizeof(protocol));
  memset(path,0,sizeof(path));
  port=0;

  bool r=devicespec_parse(argv[1],
                          devicename,
                          protocol,
                          path,
                          &port);

  if (r==true)
    {
      printf("Device Name: %s\n",devicename);
      printf("Protocol: %s\n",protocol);
      printf("Path: %s\n",path);
      printf("port: %d\n",port);
    }
  else
    printf("Invalid devicespec.\n");

  return r;
}

-Thom

[damosan]

2 hours ago, tschak909 said:

...cleanly strtok()...

strtok() is not your friend.  It's a man, living in a van down by the river, promising candy to everyone who walks by.

 

sscanf() is also not your friend but it's more like that uncle you only see at Christmas e.g.  try sscanf( path, "N%c:%s:%s:%s", bucket, protocol, path, port )

 

For this stage though it'll be fine - the final product will want a real tokenizer for this.

 

Also I'd suggest that all protocols follow the same format?

 

N:HTTP://sss:p

N:TCP://sss:p

N:UDP://sss:p

 

It's strange but make parsing easier (and safer) and it's ok if we deviate a bit (for the TCP/UDP).

Edited February 4, 2020 by damosan
Additions.

[tschak909]

nope, refuse to use sscanf, because the input stream isn't always the same.

-Thom

[+DjayBee]

17 minutes ago, damosan said:

Also I'd suggest that all protocols follow the same format?

 

N:HTTP://sss:p

N:TCP://sss:p

N:UDP://sss:p

Omit the slashes completely.

Since you parse all colons anyway, they make no sense. And it becomes easier on the Atari's side. 

Edited February 4, 2020 by DjayBee

[damosan]

21 minutes ago, tschak909 said:

nope, refuse to use sscanf, because the input stream isn't always the same.

-Thom

 

For common services the URIs/URLs are all basically the same (HTTP, FTP, HTTPS, etc.). 

 

We can do the same for TCP and UDP.  Instead of "N:TCP:3000" we can do "N:TCP:localhost:3000" which is what people typically do anyway.  As far as traditional TCP/UDP comms what doesn't fit into that standard 4-part string?

 

 

 

[tschak909]

1 hour ago, DjayBee said:

Omit the slashes completely.

Since you parse all colons anyway, they make no sense. And it becomes easier on the Atari's side. 

Some of the bits on the ESP need the slashes.

 

Keep in mind as well, that there is a command to set a URL prefix, so you can set a base URL and not have to type it each time.

 

I am _really_ wary of using sscanf.

 

-Thom

[tschak909]

I have posted documentation for the CONFIG program that boots cold start, if anyone wants to peruse and comment:

https://github.com/FujiNetWIFI/atariwifi/wiki/CONFIG-Users-Guide

 

[damosan]

Relocator update...

 

It's successfully checking for a device on boot.  The ultimate goal with this is to have a covers-all-non-sparta-dos environments but we'll see once I get more of the driver in there and test with other DOS environments.  Right now the driver is taking 631 bytes of RAM but 500 bytes of that are set asides for input/output buffers (if needed - can be reduced/removed).  I plan to implement the rest of the CIO funcs this weekend.

 

At present I show the new MEMLO once relocation is complete, the number of instruction fixes performed, and finally the number of bytes reclaimed after the assembly INIT function is called.  

 

I'm running DOS 2.5.

 

Edited February 7, 2020 by damosan

[tschak909]

2 minutes ago, damosan said:

Relocator update...

 

It's successfully checking for a device on boot.  The ultimate goal with this is to have a covers-all-non-sparta-dos environments but we'll see once I get more of the driver in there and test with other DOS environments.  Right now the driver is taking 631 bytes of RAM but 500 bytes of that are set asides for input/output buffers (if needed - can be reduced/removed).  I plan to implement the rest of the CIO funcs this weekend.

 

I'm running DOS 2.5.

 

awesome

-Thom

[Kyle22]

SDX compatibility would be nice. SDX has native relocatable drivers that can even go into extended RAM.

 

Edit: You can write one. I did. (myide.sys)

Read the Docs, Manual and the Toolkit.

Edited February 7, 2020 by Kyle22

[tschak909]

The CIO driver should work with SDX as is, with that said...

 

....SDX will ultimately need its own kernel drivers, all doable.

 

-Thom

[Kyle22]

1 minute ago, tschak909 said:

The CIO driver should work with SDX as is, with that said...

 

....SDX will ultimately need its own kernel drivers, all doable.

 

-Thom

Cool!

:)

 

[damosan]

2 minutes ago, Kyle22 said:

SDX compatibility would be nice. SDX has native relocatable drivers that can even go into extended RAM.

 

Yeah - the work I'm doing is for everything else OTHER than SDX (hopefully).  I've done some reading on how SDX does things you can expect relocatable code and loading into high memory instead of the dynamic MEMLO hack that is going on right now.

[leech]

I am legitimately excited for this.  May be able to replace my need for having a bunch of crap hanging out the back of my Atari for printing and disks, etc!

I really need to finish trying to learn some codung skills, and maybe get together some coders and we could write the first 8bit MMO.  ?

[damosan]

11 hours ago, leech said:

I am legitimately excited for this.  May be able to replace my need for having a bunch of crap hanging out the back of my Atari for printing and disks, etc!

I really need to finish trying to learn some codung skills, and maybe get together some coders and we could write the first 8bit MMO.  ?

That would be fun to work on actually.  A top/down Ultima style display but with 1000 people stomping through the world.  Having it be server based also makes the client a lot easier in that you don't have to worry about coding a story into the client - the server manages the story.  It would also be kind of cool to have a bank of 8bits acting as a clustered server to drive it though a Python script on a Linux or Windows machine would probably do just as well.

 

Mmmmm...that'd be slick.

 

As an aside I was working on a Post-Apocalypse Ultima clone two years ago - I got the basics done but stopped working on it.  I had planned to target the 130XE so I could store gobs of data and code additional gameplay stuff - as well as a very generic API to support loadable content moving forward (which would be easy to do with CC65 and banking/overlay support).

 

Anyway...back to the Fuji...

[leech]

1 minute ago, damosan said:

That would be fun to work on actually.  A top/down Ultima style display but with 1000 people stomping through the world.  Having it be server based also makes the client a lot easier in that you don't have to worry about coding a story into the client - the server manages the story.  It would also be kind of cool to have a bank of 8bits acting as a clustered server to drive it though a Python script on a Linux or Windows machine would probably do just as well.

 

Mmmmm...that'd be slick.

 

As an aside I was working on a Post-Apocalypse Ultima clone two years ago - I got the basics done but stopped working on it.  I had planned to target the 130XE so I could store gobs of data and code additional gameplay stuff - as well as a very generic API to support loadable content moving forward (which would be easy to do with CC65 and banking/overlay support).

 

Anyway...back to the Fuji...

Oooh, what a tease...  I'd totally pay for some VPS to host a server side for that.

But yeah let's not get too far off topic, looking forward to this device (may get a few for each of my 8bits).

[tschak909]

Current status:

 

@mozzwald is currently designing the next revision of the ESP32 hardware, adding changes requested by users.

 

@jeffpiep is fleshing out the filesystem classes in the PlatformIO version of the code, getting it ready so that the same functions used to access TNFS shares, also access the local SD card on the Ice Cream Sandwich.

 

@tschak909 (this is a long one...)

 

I am currently on the third attempt to rewrite the disk image browser in multilator, so that it utilizes a directory cache. Why? Because there are no seekdir/telldir/rewinddir functions to re-position the directory handle passed back when a directory is opened for viewing. This is critical because I want to be able to page through multiple pages of directory efficiently.

 

So the solution is to implement a "directory cache" corresponding of three files temporarily stored on the SPIFFS:

 

* A name cache, containing the strings of each directory entry stored end to end

* index - containing entries for the offset and length of each directory entry name, as well as the file type returned from stat()

 

This directory cache only lasts as long as the directory is open (meaning, as long as the directory browser is open), after which it is deleted from SPIFFS.

 

With this, a directory seek command can be implemented which can position a cursor along this index, after which the next read directory entry will return entries from that point.

 

16 entries will be displayed on page, making it very easy to calculate page to offset.

 

and since the stat is also returned, we can cleanly distinguish folders.

 

if I can just get this written. the difficulty lies in code inter-dependence, making the ability to test each piece individually somewhat limited. 

 

-Thom