Jump to content

Photo

Doom networking / UART bug


83 replies to this topic

#51 Welshworrier OFFLINE  

Welshworrier

    Chopper Commander

  • 249 posts

Posted Sat Jun 4, 2016 5:41 AM

The power of maths compels you!
If you have a parabolic flight where the rate of descent is different you will end up at different end points when you hit the ground level. I.e. the values of x and y will be different. The different multiplication factor I put it in are necessary because the vbls value vary between 4 and 8. So you divide each of the old values by 4 and then multiply by the vbl to get the value you really need.

I would put money on that being the fault.

#52 Welshworrier OFFLINE  

Welshworrier

    Chopper Commander

  • 249 posts

Posted Sat Jun 4, 2016 6:06 AM

Or, using a simple example to demonstrate. If you were at 0,0,10 and moving forward at momentums of x=0, y=2, z= -1. With a vbl of 4 you would be 2*10 forward when you hit the ground. If vbl was 8 then you would end up 2*2*10 forward when you hit the ground. If you modify the z momentum by the vbl as well though you would also end at 2*10

#53 Songbird OFFLINE  

Songbird

    Stargunner

  • Topic Starter
  • 1,561 posts

Posted Sat Jun 4, 2016 6:37 AM

It's easy enough to try, but I need to fix the basic soft reset first. Even after I corrected the inbytes index to:

/* */
/* check for consistancy error */
/* */
 errorCounter--;
 
 
if (errorCounter == 0)
 {
  errorCounter = 50 * vblsinframe;
  injectError = 1;
 }
 
if ((inbytes[4] != outbytes[4]) || injectError)
	{
		injectError = 0;
		netErrors++;
		PrintNumber (15,13,netErrors);
		
		otherconsoleplayer = consoleplayer ^ 1;
		
		players[otherconsoleplayer].mo->x = 
					(inbytes[6] << 24) |
					(inbytes[7] << 16) |
					(inbytes[8] <<  8) |
					(inbytes[9]);

		players[otherconsoleplayer].mo->y = 
					(inbytes[10] << 24) |
					(inbytes[11] << 16) |
					(inbytes[12] <<  8) |
					(inbytes[13]);
	}

It still doesn't work. I added the injectError and errorCounter variables so I could semi-randomly inject consistency errors. Still seeing the same behavior: as soon as it triggers the first time, it triggers every frame after that. And this is with both players just standing still at the beginning of the level, so no z issues to worry about.

 



#54 Welshworrier OFFLINE  

Welshworrier

    Chopper Commander

  • 249 posts

Posted Sat Jun 4, 2016 7:07 AM

I take it you defined error counter as a static otherwise it will be reset whenever you enter the routine. The decrement should also take place after the trigger set.

#55 Songbird OFFLINE  

Songbird

    Stargunner

  • Topic Starter
  • 1,561 posts

Posted Sat Jun 4, 2016 7:12 AM

I made errorCounter a global and set it to 100 each time the network is initialized.

 

Maybe later I will try spitting out more variables onscreen just to be sure I'm not missing something simple. Even just looking at the incoming x,y pair versus the existing x,y pair would be curious, in case something is off.

 

Also, vblsinframe is displayed in the debug info onscreen, and I've seen it vary from 3 to 7. It definitely can drop below 4, so that may affect your proposed z calculations.

 

Edit: I found the error. I was using logical && instead of bitwise & in my original outbytes set up. I just got rid of them altogether, because they weren't really necessary. Things are looking much better now. Watch for an update soon. :D


Edited by Songbird, Sat Jun 4, 2016 1:30 PM.


#56 Songbird OFFLINE  

Songbird

    Stargunner

  • Topic Starter
  • 1,561 posts

Posted Sat Jun 4, 2016 4:12 PM

Test #6: doom_n06.abs

Changes:

  1. Changed serial packet size from 6 bytes to 15 bytes.
  2. Corrected consistency byte generation. If consistency check fails, resync player x,y coordinates based on received serial packet.
  3. Added checksum byte at the end of the serial packet. If checksum byte does not match, go through the regular NETWORK ERROR display and restart the level.

Details:

 

Based on my experiments, I have never observed the infamous UART serial port bug on Doom. I don't think it's ever happening in practice, even at the 115200 speed which is the default speed for the game. I have confirmed that bytes are going round trip across the serial port with no issues. I have played multiple hours of Doom over the past week to test various experiments, and once I started checking for checksum failures, comparing player x,y versus received x,y, and so on, I have never seen a shifted byte received from the serial port.

 

So, what is the error? It appears to be a consistency error which originates from some unknown point in the code. (Welsh speculates it may be because the player z coordinate is not being kept in sync with the serial port vblsinframe variable.) The symptom of the error is that the player x,y coordinates get out of sync between the two consoles, so neither console really knows where the actual player locations are, which causes all kinds of badness. This is why vanilla Jaguar Doom just resets the level if this ever happens.

 

The fix is quite simple: add the player x,y coordinates to the serial packet. If the consistency check fails, instead of resetting the level, just move the player to the correct location. This should be fairly safe in practice, unless the real consistency error is causing something more catastrophic which I have not observed yet.

 

I tested this fix by injecting errors in the code. I forced the consistency check to fail semi-randomly during gameplay, and watched some global variables to confirm the right behavior. I was able to inject dozens of errors while playing without any detrimental side effects. The real consistency error is actually quite hard to encounter since I corrected the consistency byte generation; I have easily played several 30-60 minute sets of the game without ever seeing the real consistency error.

 

For bonus points, I could improve the checksum handling (which currently just does the level reset) to use the ACK byte and packet retry loop which I had in a previous experiment. That would be the optimal solution, just in case the UART bug ever manifested, but it appears to be so rare that I haven't bothered to incorporate it into the code. Since I am providing both versions of the code in this thread, it would be easy for someone to merge them if desired.

 

Result:

 

Doom networking appears to be fixed, however extensive testing is needed with two active players to confirm the real error behaves the same as the injected error. I'm satisfied with the results, and consider the fix complete. Game on! :D

 

Special thanks to user Welshworrier, who caught my coding mistakes as well as provided excellent guidance on how to handle protocol errors and Doom synchronization. :)

Attached Files


Edited by Songbird, Sat Jun 4, 2016 4:33 PM.


#57 Welshworrier OFFLINE  

Welshworrier

    Chopper Commander

  • 249 posts

Posted Sat Jun 4, 2016 5:15 PM

So it now works as post 41 ;)

Congrats on that, glad it's now sorted, I might even buy a jag some time and get to play it :)

The vblsinframe discrepancy comes down to some interesting code BTW, the definition of the variable says it's range is 4 to 8 and code then checks to see if these ranges are exceeded and limits. Well it's supposed to :) the code surrounding the minimum limit has had an if 0 surrounding it to disable that part of the code.

I searched for where the x,y,z positions are actually updated and was surprised to see it's all done in dsp so identifying the point of failure is going to be a bit more interesting.... Though I did like the numerous "f*cking DSP" comments.

Edited by Welshworrier, Sat Jun 4, 2016 5:20 PM.


#58 walter_J64bit OFFLINE  

walter_J64bit

    Quadrunner

  • 5,180 posts
  • Location:Goldsboro NC

Posted Sun Jun 5, 2016 3:40 AM

That's one part of "DOOM+ bucket list" checked off.  :D  Now how I'm going to play Jag DOOM with this patched network? :spidey: Thank's Welshworrier and Carl Forhan!  :thumbsup:  :thumbsup: 



#59 Welshworrier OFFLINE  

Welshworrier

    Chopper Commander

  • 249 posts

Posted Sun Jun 5, 2016 3:56 AM

Funnily enough it pretty much confirms that the fault was with doom all the time and not the jag. So Atari were right... Unless the DSP was occasionally messing up the positions of course...

The solution of sending more data to fix a networking problem on a suspect serial line is somewhat ironic :)

Edited by Welshworrier, Sun Jun 5, 2016 4:17 AM.


#60 Welshworrier OFFLINE  

Welshworrier

    Chopper Commander

  • 249 posts

Posted Sun Jun 5, 2016 4:33 AM

That's one part of "DOOM+ bucket list" checked off.  :D  Now how I'm going to play Jag DOOM with this patched network? :spidey: Thank's Welshworrier and Carl Forhan!  :thumbsup:  :thumbsup:

You could get two serial to network converters at about 50 dollars apiece. You could then play across the internet if you wanted (as long as the timeouts are okay, probably best to drop to 38400 for that though)

Edited by Welshworrier, Sun Jun 5, 2016 4:44 AM.


#61 TXG/MNX OFFLINE  

TXG/MNX

    River Patroller

  • 3,633 posts

Posted Sun Jun 5, 2016 11:35 AM

Now you fixed 2 player Doom would your code also support more players like aircars :-)

 

Will test it tomorow to see if the error doesn't occur on my consoles (I got PAL systems) I gues you only tested on NTSC models.

 

Can you tell me which type jag's you had for test K or M models or both (mixed)

 

Last thing, DOOM+ would be nice, this time a way to load new levels from CD...

 

But for now I gonna play DOOM again, It always bugged me the game crashed...



#62 omf OFFLINE  

omf

    Dragonstomper

  • 780 posts

Posted Sun Jun 5, 2016 12:11 PM

Now you fixed 2 player Doom would your code also support more players like aircars :-)

 

Will test it tomorow to see if the error doesn't occur on my consoles (I got PAL systems) I gues you only tested on NTSC models.

 

Can you tell me which type jag's you had for test K or M models or both (mixed)

 

Last thing, DOOM+ would be nice, this time a way to load new levels from CD...

 

But for now I gonna play DOOM again, It always bugged me the game crashed...

never bothered me, of course not having any one to play the game with helped with that



#63 TXG/MNX OFFLINE  

TXG/MNX

    River Patroller

  • 3,633 posts

Posted Sun Jun 5, 2016 12:12 PM

When you're in the Netherlands let me know we can play together :-)

Sent from UMI hammer with Tapatalk

#64 Songbird OFFLINE  

Songbird

    Stargunner

  • Topic Starter
  • 1,561 posts

Posted Sun Jun 5, 2016 8:25 PM

I had one K Jaguar and one M Jaguar in my dual test station.

 

Loading new levels from CD would not be practical. You'd need a new Doom cartridge in the first place ,or some other way to load new boot code onto the Jag with a vanilla Doom cart installed. Then you'd also be stuck with the tiny Jaguar RAM of about 1.75MB after the Doom code is loaded. The superior solution would be to have a SD cart where it would be easy to distribute new levels. :)

 

In theory you could expand the same simplistic networking to more players. Once all the consoles negotiate for player 1, 2, 3 and 4, then they just take turns in order sending and receiving data packets. There's plenty of UART bandwidth to handle this -- I estimate only about 12Kbps needed -- but I'm not sure if it would be too laggy since none of the four consoles could advance until all data was received. But I have no way to test for example a four-player network, so I won't be coding this up any time soon.

 

 

Now you fixed 2 player Doom would your code also support more players like aircars :-)

 

Will test it tomorow to see if the error doesn't occur on my consoles (I got PAL systems) I gues you only tested on NTSC models.

 

Can you tell me which type jag's you had for test K or M models or both (mixed)

 

Last thing, DOOM+ would be nice, this time a way to load new levels from CD...

 

But for now I gonna play DOOM again, It always bugged me the game crashed...



#65 sh3-rg OFFLINE  

sh3-rg

    River Patroller

  • 3,396 posts
  • doge + tie = dothemath
  • Location:BOLTON, England

Posted Mon Jun 6, 2016 1:51 AM

welshworrier?

welshwizard more like.

#66 CyranoJ OFFLINE  

CyranoJ

    Quadrunner

  • 5,548 posts
  • RAPTOR in LOCAL
  • Location:Adelaide, SA

Posted Mon Jun 6, 2016 2:04 AM

"Guy with no Jaguar fixes Doom"

 

"Guy with No Jaguar or CD helps beat the encryption"

 

It really is in the fumes, because it doesn't seem to be inside the plastic :)



#67 walter_J64bit OFFLINE  

walter_J64bit

    Quadrunner

  • 5,180 posts
  • Location:Goldsboro NC

Posted Tue Jul 5, 2016 3:52 AM

I would like to know if anyone has tried networking with the fixed doom and the doom that isn't fixed?


Edited by walter_J64bit, Tue Jul 5, 2016 3:56 AM.


#68 Saturn OFFLINE  

Saturn

    River Patroller

  • 3,588 posts
  • Pillar of the Jaguar Community
  • Location:TX

Posted Tue Jul 5, 2016 11:18 AM

Wow, that's some incredible work.

N00b question (more like from someone who doesn't understand how to make this work), can anyone post a flashable/usable rom or file to put on a Skunkboard? I'd like to test this out but no idea how to take the 107kb file in post 56 and combine it with a wad that I don't have.

Any help appreciated :cool:



#69 Songbird OFFLINE  

Songbird

    Stargunner

  • Topic Starter
  • 1,561 posts

Posted Tue Jul 5, 2016 11:52 AM

I would like to know if anyone has tried networking with the fixed doom and the doom that isn't fixed?


You can't do that, because the fixed Doom sends and receives a different length serial packet than the original Doom. They are completely incompatible.

#70 omf OFFLINE  

omf

    Dragonstomper

  • 780 posts

Posted Wed Jul 6, 2016 5:09 PM

Wow, that's some incredible work.

N00b question (more like from someone who doesn't understand how to make this work), can anyone post a flashable/usable rom or file to put on a Skunkboard? I'd like to test this out but no idea how to take the 107kb file in post 56 and combine it with a wad that I don't have.

Any help appreciated :cool:

a simple matter with a hex editor, the wad begins iwad starts at 40000, this can be injected to a rom of carls work at that offset with a hex editor

Attached Files


Edited by omf, Wed Jul 6, 2016 5:11 PM.


#71 walter_J64bit OFFLINE  

walter_J64bit

    Quadrunner

  • 5,180 posts
  • Location:Goldsboro NC

Posted Sat Jul 9, 2016 12:45 PM

a simple matter with a hex editor, the wad begins iwad starts at 40000, this can be injected to a rom of carls work at that offset with a hex editor

omf ROCKS!



#72 Saturn OFFLINE  

Saturn

    River Patroller

  • 3,588 posts
  • Pillar of the Jaguar Community
  • Location:TX

Posted Mon Aug 1, 2016 8:13 PM

You could get two serial to network converters at about 50 dollars apiece. You could then play across the internet if you wanted (as long as the timeouts are okay, probably best to drop to 38400 for that though)

 

I want to try this out but with the many kinds of serial to network converters out there, is there a specific brand or model that would be recommended?

 

Edit

Ha, nevermind. Guy on another forum who did this years ago recommended this cheap one, as it was said to have worked easily enough :cool:



#73 Saturn OFFLINE  

Saturn

    River Patroller

  • 3,588 posts
  • Pillar of the Jaguar Community
  • Location:TX

Posted Sun Aug 14, 2016 8:50 AM

Notes from a post a few minutes ago at another forum:

 

 

I'm testing co-op right now on level 7 but don't have a 2nd person available to help genuinely fight through the stage. I'll be creeping along shortly switching controllers. I got this network up and running at 8:00AM and will see what happens with the sync and report any findings. I know it's early yet but no problem for 20 minutes so far :)

Playing "I'm a wimp" btw.

 

Edit:
About 9:30AM and still milling about on Area 7. Have both characters in the elevated spot where the yellow key is since it appears to be safe from enemies. No sync issues so far with the players. Only things I've noticed is that when one player has their screen switched to the map, the other player experiences slow down. Works both directions but once back in the game it goes away completely. Idk if that's always been the norm though.
Also, in the area where the yellow key is, the lighting effect of dimming/brightening seems to be off by a hair. One player, on the cart that had actually made it to Area 7 and apparently set the tone for what the Area 1 capable cart would do, has the lighting effect a split second behind. That may be 'normal' as well but thought I'd throw it in.

 

Edit:
At 9:43 the player I was using crashed and the attempting to connect screen came up. Never reestablished the network and the other console froze.
Hit Option on the console I was using, restarted and stayed on the connecting screen. Other console never changed.
I'd left the other player where the yellow key card was and ventured around the map until I met locked doors that needed blue and red key cards to open. Far south on the map is where/when it failed. Probably the most distance I'd had between players so far if that means anything.
Too bad but it's a LOT better than before.



#74 Songbird OFFLINE  

Songbird

    Stargunner

  • Topic Starter
  • 1,561 posts

Posted Sun Aug 14, 2016 2:19 PM

(The below response assumes the previous post was from someone testing the last networking fix I published for Doom. See up-thread for details.)

 

That's interesting but not completely surprising. The way I left the code was this:

  1. Consistency check failures (player coordinates don't match) result in a "forced sync" of player coordinates. Gameplay always continues. This was happening fairly frequently.
  2. Checksum failures (UART glitch) display the "network error" screen and reset the network. I did not observe this in several 60+ minute sessions.

Basically, I did nothing to fix the infamous UART glitch, because that's not why Doom was failing so frequently. It was failing because the player coordinates were note being kept in sync. So, it is entirely possible that if you play long enough, you'll hit the actual UART glitch.

 

The simplest way to fix this would be to combine the "forced sync" with the "handshake and retry" method I posted upstream. The odds of two UART glitches happening back-to-back after a retry would be very slim, and you could always up the retry count to 3 or 4 to make it even less likely. Since it only happens in an error path, it wouldn't slow down 99.99% of the game.

 



#75 Saturn OFFLINE  

Saturn

    River Patroller

  • 3,588 posts
  • Pillar of the Jaguar Community
  • Location:TX

Posted Sun Aug 14, 2016 2:30 PM

I was using the latest fix, I believe. The one that OMF upped the rom of.

Replayed in Area 7 maneuvering one player all over the map, leaving the other guy at the starting position. It was about 20 minutes into the game when it crashed at the same location near the southernmost part of the map that was accessible at the time. Haven't tried it again yet. Been too busy in single player mode blazing through levels.

I'll hit it again in a little while and see if it happens a third time in the same spot.






0 user(s) are browsing this forum

0 members, 0 guests, 0 anonymous users