Jump to content
IGNORED

FujiNet bridge to Altirra, there may be a way(tm)


tschak909

Recommended Posts

#FujiNet is a complex beast, for sure. It has tons of firmware, and would need a full ESP32 emulation, or high level equivalents of everything in the firmware. This is infeasible, and @phaeron has admitted as much. 

 

But I was talking to @48kRAM tonight, and he posed a question: What if there could be a simple UDP bridge that bridged over SIO to a real FujiNet? This is possible, because the FujiNet can run entirely stand-alone, not connected to an Atari, only needing power. It could be placed into a mode where it would reflect the SIO traffic to UDP packets to the FujiNet and back. So while you would need a FujiNet device, they're only $65, and you wouldn't need a physical Atari to run it on. It would allow for software development to take place, very easily, and within the confines of emulation, where you have _EXCELLENT_ debugging facilities.

 

@phaeron is the python device server stuff documented? Could it be used to prototype something for this? Thoughts?

 

-Thom

 

  • Like 7
Link to comment
Share on other sites

@phaeron am trying to think of the best way to implement this. FujiNet does a TON of devices, can we simply inject SIO packets into the emulator for processing? or will I need to define individual SIO devices?

 

edit: looks like i need to do rawsio, and have a proxy program running on the same system.... but don't see how I can assert command the same way I can e.g. do proceed and interrupt (I do need to assert proceed for N: operations)

 

-Thom

 

Edited by tschak909
  • Like 1
Link to comment
Share on other sites

Chewing through my brain, working on how the protocol mechanism would work:

 

Basically, you can't just reflect the SIO traffic, and be done with it, because of ACK/NAK timing, too much latency.

 

So, the emulator plug-in (custom device in terms of Altirra) would need to know which devices to deal with, ahead of time, and their commands (to properly ack/nak). So you need a packet type to indicate both valid device and valid command IDs for each device that fujinet exposes. call this packet type 0.

 

Packet type 1, would be a command packet, and would consist of the 5 byte command frame.

 

Packet type 2, would be the data payload

 

Packet type 3, would be the PROCEED line 0 low, 1 high.

 

Packet type 4, would be the same for the INTERRUPT line

 

Packet type 5, would be the same for the CLKI transition (if enabled)

 

Packet type 6, would be the sme for the CLKO transition (if enabled)

 

So when the Atari asserts command for a device that we're intercepting, the custom device will immediately determine if it needs to ACK or NAK, and do it within the proper timing constraints for SIO, giving the fujinet time to assemble its requisite payload packet, and send it to the bridge server, which altirra can then deal with.

 

-Thom

 

 

 

 

 

  • Like 2
Link to comment
Share on other sites

9 hours ago, tschak909 said:

@phaeron am trying to think of the best way to implement this. FujiNet does a TON of devices, can we simply inject SIO packets into the emulator for processing? or will I need to define individual SIO devices?

 

edit: looks like i need to do rawsio, and have a proxy program running on the same system.... but don't see how I can assert command the same way I can e.g. do proceed and interrupt (I do need to assert proceed for N: operations)

You don't assert the command line, that's from the computer to devices. The custom device receives events when a valid SIO command frame is received (high-level API) or when the SIO command line is toggled (low-level API). This would only be required for an SIO bus master, which would replace the emulated computer for 10502PC-like operation. This is not currently supported. (The Indus GT can also do this, but this is not currently emulated.)

 

3 hours ago, tschak909 said:

Basically, you can't just reflect the SIO traffic, and be done with it, because of ACK/NAK timing, too much latency.

Timing within the emulation is fully virtualized and you can emulate any required timing with cycle accuracy. Any time taken by the custom device doesn't affect emulation timing and can only at most cause jank in the emulator's frame timing. The only timing you have to worry about is on your side with any devices you are communicating with. It is true that the actual timing is too tight for straight mirroring with an actual SIO devices, but in your case you control the device and can relax the protocol timing as needed.

 

3 hours ago, tschak909 said:

So, the emulator plug-in (custom device in terms of Altirra) would need to know which devices to deal with, ahead of time, and their commands (to properly ack/nak). So you need a packet type to indicate both valid device and valid command IDs for each device that fujinet exposes. call this packet type 0.

You can do this either way, as emulation time is stopped while the custom device handler runs and this can include synchronous calls to the server. I have written custom devices that do blocking server calls for memory mapped I/O; it's not fast and it can cause the emulator to drop below 60 fps if the server is slow, but it doesn't affect what the emulated program sees. Events coming from the simulation are timestamped either explicitly (supplied) or implicitly (read from the global clock), and incoming events can be timed by having a worker thread sleep for the required delay.

 

Note that currently to use the high-level SIO command handling, the device IDs and commands must be statically declared in the custom device description. Device IDs can use a range, but commands can't. The reason for this limitation is because the custom device API is simplified and uses a declarative model, while the internal API polls devices. This avoids having the script VM be in the critical path for SIO command processing, though it could be extended. Raw SIO can of course handle any command, but then you need to implement the SIO protocol manually.

 

3 hours ago, tschak909 said:

Packet type 5, would be the same for the CLKI transition (if enabled)

 

Packet type 6, would be the sme for the CLKO transition (if enabled)

Custom devices can't manually sense or clock the SIO clock in/out lines. For SIO clock in, Altirra's internal device interface only allows setting an external clock rate (required by MIDImate) and specifying whether synchronous receive is needed (required by TOMS Turbo Drive), but this is not exposed to custom devices. For SIO clock out, devices don't receive actual signals, they receive the transmission rate.

 

3 hours ago, tschak909 said:

So when the Atari asserts command for a device that we're intercepting, the custom device will immediately determine if it needs to ACK or NAK, and do it within the proper timing constraints for SIO, giving the fujinet time to assemble its requisite payload packet, and send it to the bridge server, which altirra can then deal with.

You might encounter some awkwardness with the return path since asynchronous server events haven't been exercised much. In particular, there isn't much in the way of synchronization primitives for worker threads, so you may need to get creative with how you suspend and resume the SIO protocol processing while waiting for the server -- in the worst case you can launch and join() a dummy thread, then interrupt() it from the network handler to resume processing. The script VM supports condition variables but they are not exposed to script.

 

  • Like 1
Link to comment
Share on other sites

  • 6 months later...

@phaeron I started to play with this. Found your sampledevices and deviceservers (+ Altirra help) very helpful to make first steps. Thanks for it!

 

I am dealing now with serial port speed change and how to detect it. Altirra always provides proper output bytes to custom device - provided output does not depend on speed.

On FujiNet side the toggle between standard speed and high speed is based on detecting checksum errors in received frames - but this does not happen with custom device.

Do you have some idea how to detect the serial output speed change in custom device? Then it would be possible to notify the receiver to toggle its speed too.

 

Jan

  • Like 1
Link to comment
Share on other sites

2 hours ago, apc said:

@phaeron I started to play with this. Found your sampledevices and deviceservers (+ Altirra help) very helpful to make first steps. Thanks for it!

 

I am dealing now with serial port speed change and how to detect it. Altirra always provides proper output bytes to custom device - provided output does not depend on speed.

On FujiNet side the toggle between standard speed and high speed is based on detecting checksum errors in received frames - but this does not happen with custom device.

Do you have some idea how to detect the serial output speed change in custom device? Then it would be possible to notify the receiver to toggle its speed too.

After recv_raw_byte(), the cycles per byte is in the thread-local $aux variable. For a standard 19200-baud frame, it'll be 94.

 

  • Like 4
Link to comment
Share on other sites

  • 1 month later...

I am trying to debug and understand some timing issues.

@phaeron I have a dilemma, better to use event handler, like "sio_command_changed" or separate thread with loop doing $sio.wait_command(); ... $sio.wait_command_off(); ... ?

To be able to use $timestamp I switched from thread/loop to event handler... not sure is there any significant difference between above two methods. Event handler blocks the emulator execution, thread based handler does not?

 

Jan

 

  • Like 1
Link to comment
Share on other sites

3 hours ago, apc said:

I am trying to debug and understand some timing issues.

@phaeron I have a dilemma, better to use event handler, like "sio_command_changed" or separate thread with loop doing $sio.wait_command(); ... $sio.wait_command_off(); ... ?

To be able to use $timestamp I switched from thread/loop to event handler... not sure is there any significant difference between above two methods. Event handler blocks the emulator execution, thread based handler does not?

$timestamp should work in either case, it's set whenever a thread resumes execution.

 

Both methods will block execution, as emulation is single-threaded due to synchronization and determinism. All threading in the script VM is cooperative and no other script threads or parts of the emulation are running while a script is executing.

 

The main difference between using an event handler and a thread is that event handlers can't do anything that requires emulation to pass, so they can't execute asynchronous calls like send_raw_byte(). The event handler has to delegate to a thread to do this. However, the event handler does allow you to run immediate code without spinning up a thread just to wait on the event, which is handy for cases like having the assertion of the command line interrupt any existing command processing. You can do that with a watchdog thread, but the thread will essentially just wait in a loop sitting on the command line. Conversely, waiting on the command line in the same thread that is doing the command processing is more natural when emulating devices that only monitor the command line at specific points in their processing, such as disk drives that lack interrupts and poll it in their main loop.

 

  • Like 2
Link to comment
Share on other sites

33 minutes ago, phaeron said:

$timestamp should work in either case, it's set whenever a thread resumes execution

$timestamp variable was not changing when used in thread, not sure what can be wrong:

function void command_thread_handler() {
    loop {
        $sio.wait_command();
        $network.post_message($11, 0);
        Debug.log_int("CMD ON  ", $timestamp);

        $sio.wait_command_off();
        $network.post_message($10, 0);
        Debug.log_int("CMD OFF ", $timestamp);
    }
}
Quote

Both methods will block execution, as emulation is single-threaded due to synchronization and determinism.

OK.

36 minutes ago, phaeron said:

The main difference between using an event handler and a thread is that event handlers can't do anything that requires emulation to pass, so they can't execute asynchronous calls like send_raw_byte(). The event handler has to delegate to a thread to do this.

Oops, I have to fix my "network_interrupt" handler which is calling send_raw_byte() directly! (but it was working somehow). I assume to set a PIN with set_proceed() or set_command() should be fine from event handler.

 

Link to comment
Share on other sites

Wait, in sample device rverter.atdevice in "network_interrupt" handler the send_raw_byte() is called. I am confused now. Is network_interrupt handler different from other handlers as the source of the event is not emulator (but device server)?

Link to comment
Share on other sites

9 hours ago, apc said:

$timestamp variable was not changing when used in thread, not sure what can be wrong:

So, the reason for this one is a bit obscure. The issue is that many of the raw bus commands on the SIO class require raw mode to be turned on ($sio.enable_raw(true)). Without this, the waits fail, the script starts looping infinitely within the tick, and then the thread gets nuked by a runaway loop guard in the script VM (10K iterations). That's why you see the same value for $timestamp repeatedly. I need to add some way to view when this is happening.

 

8 hours ago, apc said:

Wait, in sample device rverter.atdevice in "network_interrupt" handler the send_raw_byte() is called. I am confused now. Is network_interrupt handler different from other handlers as the source of the event is not emulator (but device server)?

That's a bug, unfortunately -- the script compiler is not throwing an error as it should here. If you place this call into a nested function then it will fail the compile as that function will be tagged as async and then the function call will fail to compile. You'll see erratic behavior with an async call in this place as the function won't be reliably resumed after the async call finishes.

Link to comment
Share on other sites

1 hour ago, phaeron said:

the raw bus commands on the SIO class require raw mode to be turned on ($sio.enable_raw(true)).

I am calling $sio.enable_raw(true) in cold_reset (only there). I believe, it is running in raw mode, the waits are working. The output corresponds to SIO activities. It is not like one shot and stop (by VM). Only $timestamp is not updated.

 

Running this test...

function void command_thread_handler() {
    Debug.log("Hi");
    loop {
        $sio.wait_command();
        Debug.log_int("thread CMD ON  ", $timestamp);
        $sio.wait_command_off();
        Debug.log_int("thread CMD OFF ", $timestamp);
    }
}

event "sio_command_changed": function {
    if ($sio.command_asserted()) {
        Debug.log_int("event  CMD ON  ", $timestamp);
    }
    else {
        Debug.log_int("event  CMD OFF ", $timestamp);
    }
};

...is producing result like this. Only $timestamp in thread is not changing but waits in thread corresponds with command level changes captured by event handler.

CUSTOMDEV: Hi
CUSTOMDEV: event  CMD ON  82819965
CUSTOMDEV: thread CMD ON  46135568
CUSTOMDEV: event  CMD OFF 82828552
CUSTOMDEV: thread CMD OFF 46135568
CUSTOMDEV: event  CMD ON  82870396
CUSTOMDEV: thread CMD ON  46135568
CUSTOMDEV: event  CMD OFF 82877359
CUSTOMDEV: thread CMD OFF 46135568
...

Maybe it is expected behavior. From help:

Quote

In particular, the $timestamp variable is thread-local. In an SIO script, it always gives the timestamp of the start of script execution regardless of suspend points within the script.

 

 

1 hour ago, phaeron said:

That's a bug, unfortunately -- the script compiler is not throwing an error as it should here.

Ok, I see.

 

Edited by apc
Link to comment
Share on other sites

No, that's not expected behavior, but a bug that I fixed post-3.90 when I unified all of the thread resume queues. It works as you would expect in 4.00-test.

 

The note regarding the SIO script is specific to that case because there sequencing is done through the emulator's SIO module instead of the main scheduler -- SIO command processing works differently because there is an active command being tracked and the SIO manager can short-circuit all of the waits to accelerate a command. For a conventional worker thread $timestamp is expected to update after every async call that allows time to pass before resuming.

  • Like 1
Link to comment
Share on other sites

Happy emulating with 4.00-test39 ?

CUSTOMDEV: Hi
CUSTOMDEV: event  CMD ON  58910483
CUSTOMDEV: thread CMD ON  58910484
CUSTOMDEV: event  CMD OFF 58917924
CUSTOMDEV: thread CMD OFF 58917925
CUSTOMDEV: event  CMD ON  58976893
CUSTOMDEV: thread CMD ON  58976894
CUSTOMDEV: event  CMD OFF 58983871
CUSTOMDEV: thread CMD OFF 58983872

 

  • Like 1
Link to comment
Share on other sites

many are stuck at 3.90, windows 10 won't run on their old rigs and windows 11 won't run on dang near anything... many folks are retired and don't/won't/can't do these moves... trivial for most, impossible for many of them. This is kind of the lament voiced quietly before but they are indeed being left behind. It is what it is... since most senior living facilities don't allow the room for a full real hardware setup, emulation is key... laptops don't take up much room, but they can't afford the endless every couple/few year latest greatest equipment and OS upgrades. It's kind of a sad reality.

  • Like 1
Link to comment
Share on other sites

6 hours ago, _The Doctor__ said:

many are stuck at 3.90, windows 10 won't run on their old rigs and windows 11 won't run on dang near anything... many folks are retired and don't/won't/can't do these moves... trivial for most, impossible for many of them. This is kind of the lament voiced quietly before but they are indeed being left behind. It is what it is... since most senior living facilities don't allow the room for a full real hardware setup, emulation is key... laptops don't take up much room, but they can't afford the endless every couple/few year latest greatest equipment and OS upgrades. It's kind of a sad reality.

4.00-test only requires Windows 7 32-bit with an SSE2 capable processor, which is a 12 year old OS spec and a ~20 year old processor spec. That is the first operating system requirement bump I've ever done and not an unreasonable requirement in order to run the currently updated version of software. I already supported Windows XP not only after it had gone out of support but its two successors had also gone out of support, and up until the point that the compiler also dropped support for it. That you would compare this to Windows 11's stratospheric system requirements is unfair and rude.

 

  • Like 5
Link to comment
Share on other sites

I referred to windows 10 as well, I didn't mention XP, VISTA, or ME but let's be hyperbolic and only speak to windows 11 and XP which is also very rude... I will try to help out still... if at all possible... maybe there's a bunch of Windows 7 COA's or what not I could use... It's really not that important apparently ... after all most of these folks are just ignored and forgotten anyway. It's just a complaint from people in senior living and old peoples homes that I relayed. A world I recently have been visiting for various reasons.

 

Anyway, to be honest, I didn't like when XP support was dropped now that I think about it, nor that it's also stopped working properly on VISTA... though Millenium Edition was a real mess so that's not as big of a deal... Windows Seven complained... so maybe 8.1 or better is the way to go since 7 is a pain and probably will be dropped soon for Altirra as well anyway, Microsoft will find a way to drop that convenient check mark from the toolchain I'm sure or most folks won't want to click it with their mouse... I'll poke around and see how many 8.1 COA's I can find and do some updates ... though they are stuck in their ways as well as thoughts... Just the task bar changes are enough to cause them fits.

 

It's a difficult crowd... it's clear to see why no one would speak about it though...

 

Clearly this is somehow a sore spot, maybe I should request that all mention of it be deleted and forever be forgotten. It's a very small and dwindling group of soon to be gone folks anyway is the vibe I just got... Just a very sad position and feeling to have is all.

 

Edited by _The Doctor__
  • Like 1
  • Sad 1
Link to comment
Share on other sites

that almost sound sensible, wine may help some of them out, if it's all automatic.

 some of them are locked into only what they know and nothing new is remembered the next day... if there was a way to get wine to run on windows of old that would run altirra that would be the best, that way they could click the Atari icon and it would launch wine and then wine autorun altirra... if Atari VCS's weren't so darn expensive that would be small enough and simple enough... no it's still going to be confusing for them that OS is completely different and acts like a smart TV which they can't seem to fathom... maybe a Pi solution, put it in a case and have it do emulation... Altirra on Pi somehow might be perfect. Something other than the community TV and staring at the walls. It still confounds me how a person can't handle a new way of windows or can barely handle windows of any kind... but can use an Atari, Apple, Commodore... or DOS machine command lines like it's nothing... those things are not exactly user friendly for most of us today... The mind is a strange thing.

 

oh heck I'm derailing the thread... this could be a thread all on it's own... should flag them all and have them moved to their own thread

Edited by _The Doctor__
Should flag all of these and move them to their own thread
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...