Load ROMs over network in Stella on Retron 77?

[chrinfinity]

I'll be working on this in the next couple of weeks, and I was wondering if anyone had tried before and could give advice or pointers.

 

In addition to a modest cartridge collection I've been waiting to try out, I also have some Atari 2600 ROMs and homebrew stored on a NAS network drive, to which I am hoping to be able to navigate and load into Stella on the Retron 77, directly from the network, via one of the USB Ethernet adapters that is compatible with the Community Build firmware when set in "developer" mode.

 

My hope is to use either SMB (Windows network folder sharing) or NFS (linux network file sharing), and mount the remote folder into the Linux file system of the Retron 77, so that I can load games directly over ethernet as if they were local to the Retron 77 device.

 

Please don't suggest I can just add roms to the SD Card, I know very well that I can do that, but I want to get the network loading thing working as an extension of my hobbies. Thank you very much.

 

P.S. If I am able to get this working, I will return here and post my findings.

[Prizrak]

I love it when people going against the grain and use out of the box thinking. That's what creates things like the PlusCart, MovieCart, etc. I have a PS2 fat that has a 2tb HDD that's full and all my other games SMB shared via a nano router with a portable hard drive connected. I had two drives that I would swap out from time to time and figured this was the best method considering my needs.

Sent from my SM-N960U using Tapatalk

[+stephena]

2 hours ago, chrinfinity said:

My hope is to use either SMB (Windows network folder sharing) or NFS (linux network file sharing), and mount the remote folder into the Linux file system of the Retron 77, so that I can load games directly over ethernet as if they were local to the Retron 77 device.

It almost definitely won't work out of the box, since I don't think we compile NFS support into the kernel.  And Samba definitely isn't there at all.  Perhaps using the kernel CIFS filesystem, and mounting the share that way will work?

 

You can definitely connect to the R77 using SSH.  That's how we do testing for the most part.  File sharing is another matter, though.  Technically not impossible, but will require work on the kernel in the R77.  And the kernel is a 3.x series, so I don't know if that's recent enough to do what you want.  Personally, I'd try first with CIFS, so you can get a mount without needing to run daemons (like normal NFS or Samba would require).

[+Karl G]

Now I'm wondering if anyone has tried one of those WiFi-enabled SD cards in a R77?

[chrinfinity]

Thanks for the tip on CIFS versus SMB! I was one of those people who didn't quite know the distinction already. Based on your comment I found an article from which I was able to learn the details: https://www.varonis.com/blog/cifs-vs-smb/

I will definitely try CIFS first. If that doesn't work, I guess I'll need to look into that kernel! I have never recompiled a linux kernel before, but I see that the community build firmware has instructions for how this can be done within docker. That's very daunting, but also exciting! I have been meaning to learn more about using docker. My intuition, and educated guesses based on reading I've done in the past, is that with a bit of luck all I will need to change is the appropriate flag(s) in the build depending on if I need NFS or SMB. *nervous laughter* Hey maybe CIFS will just work on the first try

 

Oh, and regarding those WiFi enabled SD cards, I looked into that when I was doing up my GameCube (which I also finally got loading/booting games over the network as well). The problem with those cards is that they always want to work the other way around from what I need -- In that they have a local data store on the card, that they then expose against WiFi for other devices to manipulate, whereas I need the data to remain on the central network store and come remotely to the device(s) as needed.

Edited May 10, 2021 by chrinfinity

[chrinfinity]

Addendum: Uh oh, CIFS might be limited to working with SMBv1 (as opposed to v2 and v3)... I can't tolerate SMBv1 on my network due to security/ransomware vulnerability reasons. So CIFS might just have to be a "proof-of-concept" for me, even if it does work  Now I just need to wait for that USB OTG adapter to come for the retron so I can start SSHing my way to victory. I will report back.

[DirtyHairy]

Just use sshfs, I took care to make sure that works ?

 

Sorry, I forgot you need access the other way round. You should be able to get NFSv3 working though if you enable the necessary parts in the kernel and busybox configs.

Edited May 10, 2021 by DirtyHairy

[chrinfinity]

Your suggestion gives me an idea, though...

 

I wonder if I could get the Retron on startup to hit the NAS with a webhook when it boots, and trigger an sshfs from the NAS back into the Retron through automated scripting? Mount a folder that's local to the NAS into a folder in the Retron, over an automated SSH session initiated by the NAS (but triggered by the webook from the Retron booting up)

Edited May 10, 2021 by chrinfinity

[+SvOlli]

Here's what I would do: compile rsync for the RetroN. Then decide if it's better to run it as a server to write new ROMS to SD, or to have it check up upon startup on new data. When running with the right parameters (something like --filesize-only comes to mind) it's possible to have it just using a second or two to run the sync.

[DirtyHairy]

I am afraid your webhook will not work that way, as you need to run sshfs on the R77 if you want to access the files on another machine. You could enable FUSE in the kernel and build sshfs for the R77, but I think SvOlli's idea is easier and more reliable.

[+SvOlli]

...and taking a look at the build system, I suggest to use sftpserver as a blueprint:

 

- create a new directory app/rsync

- copy app/sftpserver/Makefile to app/rsync/Makefile

- get the rsync source code by running "cd app/rsync ; git clone https://github.com/WayneD/rsync.git"

  (you might want to select a release tag instead of the head version)

- now comes the hard part: adjust the app/rsync/Makefile that it runs the build process of app/rsync/rsync correctly

- in the top-level Makefile duplicate every line containing "sftpserver" and replace "sftpserver" with "rsync" in that second instance

- once you've come so far, add config files and or scripting for running rsync upon startup

Finally you can create an sd card image that supports network updating of the ROMs. Welcome to the wonderful world of embedded development.

 

Hint: you don't want to take a shortcut that will leave you with something you can't reproduce.

 

And another two cents: looking at the effort it takes to get this approach done, this might look like a lot. It is. But every other option discussed so far will result in a bigger effort with greater chance of a fail. I do embedded development for a living and for fun in my spare time (besides retro stuff), so I've gained experience in estimating efforts and chances of success.

 

@DirtyHairy: You told me that you're not a fan of the build system done by Hyperkin. Let me assure you, I've seen worse. Much worse. Done by a company who needs to get certification for their code.

[DirtyHairy]

35 minutes ago, SvOlli said:

Let me assure you, I've seen worse. Much worse. Done by a company who needs to get certification for their code.

I can relate. I earn my money as a web developer, and the levels of bad that you can see in mission-critical in-house software can be unbelievable ?

[Thomas Jentzsch]

11 minutes ago, DirtyHairy said:

...the levels of bad that you can see in mission-critical in-house software can be unbelievable ?

Amen

Edited May 11, 2021 by Thomas Jentzsch

[chrinfinity]

I wanted to give an update, my OTG usb cable/adaptor (necessary to connect the ethernet adapter to the Retron 77) got held up in Amazon stock limbo, so I had to cancel the order and order an alternative cable. This project is still top of my mind though, I'm expecting the replacement cable next week and will report back.

 

Currently based on your helpful feedback, I'm planning to see if I can make sshfs work for my purposes, and try to establish the connection automatically on boot with a script.

[chrinfinity]

Update: OTG cable arrived, and worked with an old Apple-branded USB-A to Ethernet dongle I had laying around. An internet posting said the Apple dongle might be "AX88xxx" chipset-based, which is known compatible, but regardless what chipset it is, it definitely worked on the first try so I am happy about that 

Retron 77 picked up an IP from DHCP, and I was able to SSH into the retron (so the very fine custom firmware did successfully generate an RSA fingerprint, very timely as well). I'm going to spend some time exploring and playing around in here to see what I can do before I get into setting up docker and trying to compile a new kernel. I misunderstood some earlier instructions and I thought that I was going to be able to get the retron 77 to SSH to another server, but that doesn't seem to be the case out of the box. Next step for me is more RTFM

[chrinfinity]

BEHOLD THE AWESOME AND INCOMPARABLE POWER

 OF MY NECROMANCY!

 

I am actively working on this project again!

 

Progress:

 

Tonight, I managed to set the correct flags in busybox config to enable CIFS/SMB, and recompile DirtyHairy's custom Retron77 firmware inside of the build environment docker container that he offers. @DirtyHairy

 

The busybox part of the compile works perfectly, but then the compiler is crashing inside the actual Stella emulator code. I think the issue is with Stella (not related to CIFS/SMB or busybox).

 

I tried hacking together a fix using Stella latest master, but I failed.

 

I left an issue open on the build container GitHub detailing my troubleshooting:

 

https://github.com/DirtyHairy/r77-firmware-ng-build/issues/1

 

Pretty sure once we get that solved, I'll be able to set up networking the way I want.

Edited August 29, 2022 by chrinfinity

[Fort Apocalypse]

51 minutes ago, chrinfinity said:

https://github.com/DirtyHairy/r77-firmware-ng-build/issues/1

 

 

Pretty sure once we get that solved, I'll be able to set up networking the way I want.

Hopefully this helps, and it could result in delays accessing the files, but instead of replacing all with relative paths, how about resolving the paths to canonical paths?

https://en.cppreference.com/w/cpp/filesystem/canonical

 

Note that you may need access to each parent directory on path:
https://developercommunity.visualstudio.com/t/stdfilesystemcanonical-improper-failure-on-network/927087

 

If you need to avoid file access delays from canonical path resolution, and if C++/etc. doesn't already cache it, you could create a class to call to get the path that would wrap the access to the filesystem. It could cache the path: first check the hash for a path, then if check the size of the hash and if it's small enough then store the path, otherwise just clear the hash first, then store the path, for a quick-and-dirty implementation. If you can avoid caching, I would, though. That way, in case it changes, it won't fail.

Edited August 29, 2022 by Fort Apocalypse

[+Al_Nafuur]

@chrinfinity maybe you can connect to the Public Area of the PlusStore. The API uses simple http requests.

PM me If you are interested.

[Thomas Jentzsch]

3 minutes ago, Al_Nafuur said:

@chrinfinity maybe you can connect to the Public Area of the PlusStore. The API uses simple http requests.

PM me If you are interested.

There even is an issue for this for Stella. But I doubt we would get to it anytime soon.

[+Gemintronic]

Anyone have any luck with those weird wifi capable SD cards?  Would such a solution even apply?

[+Al_Nafuur]

38 minutes ago, Thomas Jentzsch said:

There even is an issue for this for Stella. But I doubt we would get to it anytime soon.

The issue is about PlusROM support. Would be really nice to have that on the Retron77 too, but connecting to the PlusStore is something completely different 

[Thomas Jentzsch]

2 hours ago, Al_Nafuur said:

The issue is about PlusROM support. Would be really nice to have that on the Retron77 too, but connecting to the PlusStore is something completely different 

Sure, but this would be the first step required to get there.

[Thomas Jentzsch]

3 hours ago, Gemintronic said:

Anyone have any luck with those weird wifi capable SD cards?  Would such a solution even apply?

They work, but not as good and flexible as I had hoped for.

[chrinfinity]

So @DirtyHairy came to the rescue, committed and posted something to the r77 firmware repository, and the build container for the custom firmware became functional again.

 

I was able to compile custom firmware and it works! I enabled busybox config flags CONFIG_FEATURE_MOUNT_NFS=y and CONFIG_FEATURE_MOUNT_NFS=y.

 

...But my attempt to mount the shared folder keeps failing. I've tried it a few different ways and I keep getting "no such device." I'll keep trying, but I'm open to ideas

[chrinfinity]

Update: I think the kernel I compiled is missing cifs.ko. (This is my first kernal compile)

 

Working on it now.