Fixing known bugs in Handy emulator

[sage]

Cool.

Nice work.

Will you release a bugfixed binary?

[LX.NET]

I guess I could build a new version (for Windows i.e.). Would there be any interest for that?

 

Even better would be to get in touch with Keith and see what we can work out with him. I'd hate to branch his codebase. I've made other changes as well, such as getting the source code to compile under Visual Studio 2010. Not sure whether that is appreciated.

[sage]

actually I do not care so much about a windows binary

What I would like to have is something which run without problems and speed penalties unter linux.

 

But anyway:

The bugs should be fixed in dependent products like mednafen. But before I would like to test it.

 

I will try to compile my own version with turboc++

[sage]

I tested it with mednafen, everon now works.

Afetr I changed the correct src, also the sprite drawing is now working.

I did NOT check as lot of ROMs/demos for "side effects" .

Edited December 25, 2011 by sage

[+karri]

Has anybody got Mednafen to run Lynx stuff in 64-bit mode? I can get it to work in 32-bit mode but 64-bit just gives me a black Lynx screen. The mednafen help and other stuff show up as they should so I assume that the Lynx emulation is not 64-bit safe.

 

--

Karri

[+karri]

Another topic is audio stuttering. In Handy Keith used an API that allows him to stream data to the OS. This is good if the audio is known beforehand. For emulating a real Lynx we need a much more real-time approach. What about the JACK toolkit? It would be suited for low-latency audio (below 1ms latency). Could that be used?

 

--

Karri

[akator]

I guess I could build a new version (for Windows i.e.). Would there be any interest for that?

 

Yes, I would appreciate a Windows binary

[sage]

>Has anybody got Mednafen to run Lynx stuff in 64-bit mode? I can get it to work in 32-bit mode but 64-bit just gives me a black Lynx screen. The mednafen help and other stuff show up as they should so I assume that the Lynx emulation is not 64-bit safe.

 

Did you run the 32 bit verseion on a 64 bit system or a native 64 bit version?

For the later it may very well be that varaibles (pointers) chnaged size and as stuff is partly quiet lowlevel...

[Shannon]

Nice to actually see some work going into improving Handy's emulation.

[+karri]

>Has anybody got Mednafen to run Lynx stuff in 64-bit mode? I can get it to work in 32-bit mode but 64-bit just gives me a black Lynx screen. The mednafen help and other stuff show up as they should so I assume that the Lynx emulation is not 64-bit safe.

 

Did you run the 32 bit verseion on a 64 bit system or a native 64 bit version?

For the later it may very well be that varaibles (pointers) chnaged size and as stuff is partly quiet lowlevel...

 

Native 64 bit. I am also compiling my new Lynx stuff on a 64 bit Oneiric laptop. So I really need to get this fixed.

 

--

Karri

[+karri]

Another topic is audio stuttering. In Handy Keith used an API that allows him to stream data to the OS. This is good if the audio is known beforehand. For emulating a real Lynx we need a much more real-time approach. What about the JACK toolkit? It would be suited for low-latency audio (below 1ms latency). Could that be used?

 

--

Karri

 

There was some guy years ago that got in touch with me for making chip-tunes. The idea was to store the timestamps and controls the Lynx uses to control the audio chips. Then you can save the streams as chip music. Later you can write a "decoder" that emulates the real hardware and turns the sounds to playback streams.

 

It might be a good idea to record this in handy and play the audio in a separate thread using a chip-tune player. The format could be something like:

 

time address data

time address data

...

 

Where address is the hex address of the chip and data is the byte written there. Actually it might make sense to have something like:

 

struct {

unsigned short time; // Time in milliseconds since previous entry

unsigned char address; // Add $FD00 to this as all timers and audio stuff is in Mikey

unsigned char data;

} chip_tune_t;

 

Modern multitasking computers would be good in running the emulation in more than one thread.

 

We could also tackle graphics in the same way. In this way we would have completely separated graphics emulation module, sound emulation module and CPU emulation module.

 

Then we could do video streaming with good audio directly from the emulator. For playback we could write a Lynx decoder that could be used for transcoding the Lynx stream to mpeg or something else.

--

Karri

[ThomH]

Where address is the hex address of the chip and data is the byte written there. Actually it might make sense to have something like:

 

struct {

unsigned short time; // Time in milliseconds since previous entry

unsigned char address; // Add $FD00 to this as all timers and audio stuff is in Mikey

unsigned char data;

} chip_tune_t;

 

This is exactly how I've written my emulators in the past. Audio events coming from the real hardware are timestamped and queued; they're dequeued at exactly the right moment in the stream when the audio output is being generated, and audio is generated when the OS requests another audio buffer. With some simple logic to dash ahead quickly through the queued events if the output falls too far behind the input and to pause if it gets too close to catching up and suitably small buffers the latency is kept within a sufficiently reasonable range not to be noticeable.

 

I'm no expert on the Lynx sound hardware specifically but in order accurately to reproduce the source data you should technically construct the source wave at its original sampling rate and then apply a filter to reduce it to the 44.1Khz or whatever you have for output. It's DSP stuff that tends to sound complicated but one way or another you end up conceptually doing a weighted average on a sliding window and then point sampling (except that as an obvious optimisation you actually just need to work out the weighted average at the points you intend to sample). I can offer a not particularly optimised implementation of a Kaiser Bessel filter from my current emulation project if it's helpful. What that'll effectively do is what the analogue filter inside the real Lynx does — albeit that the '4 Khz' cutoff given in the public documentation is almost certainly a typo.

 

If there's anything I can do to help, I'd be more than willing.

Edited December 31, 2011 by ThomH

[+karri]

What should the language be? .NET will not work for me as I run mostly Ubuntu. C++ is pretty close I believe.

 

--

Karri

[ThomH]

The available evidence seems to be that LX.NET's emulator is a conversion of the Handy sources into Microsoft CLR managed code — so that's not going to become portable any time soon — and mednafen is just a direct lift of the Handy code. I think I'm right to say that Microsoft still support C++/CLI, which is a very close relative of standard C++ that works in Microsoft's virtual machine but I'm not sure how well catered for it is in terms of tools or whether it has the correct bindings to work in places like Windows Phone 7 and on the XBox.

 

Incidentally, I've checked the scans of the original Epyx documentation provided on this site and the 4 Khz lowpass filter figure is given in quite a few places, meaning that it's definitely not an OCR or transcription error and probably not a typo. To put things in perspective, per Nyquist that's the same as saying not to bother recording (or, with an emulator, outputting) Lynx audio at anything above an 8 Khz sampling rate because the higher frequencies just aren't there. I guess they had an eye firmly on the quality of the internal speaker.

 

It's a yucky mediafire link, but a sample of pure 4 Khz tone is here to put things in perspective. It sounds quite high, but normal human range across all age groups is considered to go up to about 20 Khz and even as someone who was the right age to be in the target audience when the Lynx came out I'm still able to discern a little above 13 Khz.

[LX.NET]

Dropping in for a few words here.

The .NET code of my emulator is indeed not very portable across different platforms other than Windows. It does run on Windows, Windows Phone 7 and above, and (with some endian-bugs) on XBox360. Since I am far more proficient in managed languages of .NET I am starting work there with intentions to work it back into the Handy codebase. Two items (EVERON bug and sprite data unpacking) have been done so far.

Also, I am working on encryption algorithms and getting my head around that. Yesterday I finished the code that allows for arbitrary 65SC02 code to be encrypted, so the Lynx can decrypt it. Read the Lynx programming forum for more info (once I post there later today).

 

Current state of .NET emulator is:

• No audio yet. I have done a port of the code in Handy, but the emulator is still too slow to keep up with the 0.25 second samples. Net effect is that the audio is stuttering. Also, it does not give any sounds on channel 1. I've read that was a bug in the Handy code (around version 0.70 or so), where Keith mentioned a fix for timer 0, because audio channel 1 was not giving any sound. Need to look into that, as I may have introduced the same bug
  
• Starting work on EEPROM support. 93C46 (Lynxman Flashcart, Lynxopoly) and KM28C64-20 (Eye of the Beholder).
  
• Graphically on par with Handy
  
• 3 small bugs remaining
  
  • randomness doesn't seem to work yet (Lynxopoly always rolls 9 with dices, and Blockout always has 2x2x1 blocks
    
  • loading of "rebuild phase" screen in Rampart has bad graphics
    
  • interlaced graphics seem more flickery/unstable than in Handy (in both it looks aweful, but a little worse in my version)
    

On the audio in particular: Keith used a small circular audio buffer connected to DirectAudio that he kept writing to. His emulation code does exactly what the audio hardware would do. Create output levels (standard or integrated audio) for a period of time and averaging over the four channels. Emulate the shift register feedback for next output value and keep doing this whenever an audio timer triggers.

The comments on the audio give me some nice suggestions to try things a little differently. It would be relatively easy to store the data of an audio channel in a different stream for playback later on. That would allow for some easy experiments with the data. I'll give it a shot as soon as I finish the encryption stuff and the EEPROM emulation (also for Handy).

[ThomH]

If I could make a feature request: is there any way the emulator could expose the debugger (and maybe simple additional functionality, like transmitting a cartridge image) via an HTTP service?

 

I'm a Mac user, so another person that falls broadly into the 'not Windows' camp, and primarily working on development tools at the minute. It'd be great if I could control the emulator from within my own tool (such as for step debugging through source code) and an HTTP approach has the advantage that it can work across a network — whether a real one or, more probably, the emulator in a virtual machine underneath whatever other OS.

 

Also as a Mac user, and someone admittedly quite ignorant about the exact limits placed on browser Silverlight, if you could get a working build there then it'd be really useful for me. I appreciate that Silverlight is being killed off as a plug-in though, so I completely understand if it's not worth expending the effort on.

[twipley]

Any new registry entries planned? If so, it would be nice for them to be specified in the changelog.

 

I guess I could build a new version (for Windows i.e.). Would there be any interest for that?

I, for one, would be interested in such a Windows-compatible binary. (EDIT: the big drawback for me in v0.9.5 always was the crackling audio.)

Edited January 6, 2012 by twipley

[Tursi]

The .NET code of my emulator is indeed not very portable across different platforms other than Windows. It does run on Windows, Windows Phone 7 and above, and (with some endian-bugs) on XBox360. Since I am far more proficient in managed languages of .NET I am starting work there with intentions to work it back into the Handy codebase.

 

I wonder, I always thought (but have not yet had any time at all to verify), that Mono (http://www.mono-project.com/Main_Page) made .NET cross-platform enough to be useful. Have you had a chance to look and see whether your code will run under that? If so, that may give you the Mac and Linux platforms as well.

[+karri]

The .NET code of my emulator is indeed not very portable across different platforms other than Windows. It does run on Windows, Windows Phone 7 and above, and (with some endian-bugs) on XBox360. Since I am far more proficient in managed languages of .NET I am starting work there with intentions to work it back into the Handy codebase.

 

I wonder, I always thought (but have not yet had any time at all to verify), that Mono (http://www.mono-project.com/Main_Page) made .NET cross-platform enough to be useful. Have you had a chance to look and see whether your code will run under that? If so, that may give you the Mac and Linux platforms as well.

 

The Lynx is a bit large to carry around. Today I play my Lynx games mainly on a PSP. It has good battery life time, goes nicely in my pocket while travelling, has WiFi, movies and music. It emulates the Lynx very well apart from the sound issue.

 

To make the emulator really useful it should be very easy to compile it for different uses.

 

1) I definitely want to have it on my PSP based on ARM CPU. No mono will be available for that one. The same is probably true for other generic hand-helds with sufficient power to emulate the Lynx.

 

2) The other very important topic is debugging and test runs. While you are writing the code you want to check small pieces at a time of how the work on the Lynx. Having a real Lynx with you all the time is not very practical. It is much nicer to be able to single-step through code in an emulator, put brakepoints, check variables. The new emulator should support all this. And also be compatible with high-level C-debugging. The cc65 suite has high level debugger support today for symbolic C-debugging. I would also like to write some kind of emulator module for Code::Blocks with proper syntax highlights and full debugging support.

 

Writing new games for a dead system like the Lynx is relaxing. The CPU + Suzy + Mickey is a very nice set of chips. It is very easy to code for it. And nobody cares if I miss my deadlines so there is no stress involved. So I really hope to be able to play around with the Lynx for a looong time still.

--

Karri

Old consoles don't die, they just get emulated.

[LX.NET]

The .NET code of my emulator is indeed not very portable across different platforms other than Windows. It does run on Windows, Windows Phone 7 and above, and (with some endian-bugs) on XBox360. Since I am far more proficient in managed languages of .NET I am starting work there with intentions to work it back into the Handy codebase.

I wonder, I always thought (but have not yet had any time at all to verify), that Mono (http://www.mono-project.com/Main_Page) made .NET cross-platform enough to be useful. Have you had a chance to look and see whether your code will run under that? If so, that may give you the Mac and Linux platforms as well.

 

Hi Tursi,

 

You have made an excellent point. I hadn't really investigated that direction yet, but it seems like a brilliant idea. I have downloaded SDL.NET already to see if that could serve as the cross-platform library for graphics and sound. If that's the case, then it is just a matter of creating mono-compatible code. Since I already use a fairly restrictive subset of .NET (using Portable Libraries to stay within the confines of both Windows Phone and XBox360) I hope to have some good chances there.

Do (any of) you know about the endianness of MacOSX and Linux distributions? I know there are a few little bugs in the switching of endianness for the sake of XBox360

 

Did a quick check on my C# code in MonoDevelop: it all compiles. Untested, but even higher hopes of making this work. I'll keep you all posted if/when I get around to doing more on this.

[Tursi]

Do (any of) you know about the endianness of MacOSX and Linux distributions? I know there are a few little bugs in the switching of endianness for the sake of XBox360

 

I don't know offhand unfortunately, but wouldn't it be the same nowadays? MacOSX is Intel-only, isn't it? Linux I dunno though I assume that little endian remains prevalent there.

 

Did a quick check on my C# code in MonoDevelop: it all compiles. Untested, but even higher hopes of making this work. I'll keep you all posted if/when I get around to doing more on this.

 

That's pretty awesome I'd love to hear it works.

[rockman_x_2002]

Do (any of) you know about the endianness of MacOSX and Linux distributions? I know there are a few little bugs in the switching of endianness for the sake of XBox360

 

Intel-based CPUs (x86, x86-64) are little-endian (including modern Macs and Linux machines).

 

Older Macs and the Xbox 360, which rely on a Power PC CPU, are big-endian, if I'm thinking correctly.

Edited January 20, 2012 by rockman_x_2002

[LX.NET]

Do (any of) you know about the endianness of MacOSX and Linux distributions? I know there are a few little bugs in the switching of endianness for the sake of XBox360

 

Intel-based CPUs (x86, x86-64) are little-endian (including modern Macs and Linux machines).

 

Older Macs and the Xbox 360, which rely on a Power PC CPU, are big-endian, if I'm thinking correctly.

 

I think you're right on that. In the meantime I have created a Mono project using MonoDevelop that compiles the unaltered C# source code for the lynx emulator. It also contains a new SDL based project for the client app. I do not have any experience with the SDL library, but did get a running game. It even runs faster than the Windows based versions!

If anyone is interested in trying out the Mono/SDL version on Linux or MacOS feel free to contact me through PM.

[sage]

Just to update this information.

I compiled it with mono under Ubuntu 11.10 and (after a few minor fixes) it worked. There are still some flaws, but basically it works.

[twipley]

Would it be feasible (and motivating) for you LX to provide a patch for the crackling audio to be fixed?

 

http://sourceforge.net/tracker/index.php?func=detail&aid=3456444&group_id=105313&atid=640866 -- just to know your thoughts about that.