Jump to content

Photo

Ahl's Benchmark?


141 replies to this topic

#26 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,261 posts
  • Location:Australia

Posted Fri Nov 30, 2007 8:27 PM

C-64s integer variables are actually pretty useless.

You only save RAM if they are referenced less than several times.

That is because the C-64 doesn't tokenize variable names in the program code - each reference takes a byte per character.

So, i costs you 2 bytes minimum each reference, where FP is only one byte.

Of course, for arrays the savings will usually outweigh the losses.

#27 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sat Dec 1, 2007 7:29 PM

Oric Atmos

Time 2:24
Accuracy : 1.04141235E-03
Randow : 3.47206211

5 GET A$:SHOOT
10 REM Ahl_s simple benchmark
20 FOR N = 1 TO 100: A = N
30 FOR I = 1 TO 10
40 A = SQR(A): R = R + RND(1)
50 NEXT I
60 FOR I = 1 TO 10
70 A = A^2: R = R + RND(1)
80 NEXT I
90 S = S + A: NEXT N
100 PRINT "Accuracy ";ABS (1010-S/5)
110 PRINT "Random ";ABS (1000-R)
120 SHOOT

The SHOOT command was added to make it easier to time.

#28 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sat Dec 1, 2007 7:49 PM

Well, as far as 8 bit machines go the current uncompiled best is around 30 seconds for the Apple IIc Plus (4MHz 65C02). The best compiled is 6 seconds for the Tandy CoCo 3 using BASIC09.

Any faster from the 6502 will probably require an after market accelerator board. I'm guessing the IIgs would be fastest but the C64 has a 65c816 board with a high clock speed.

The MSX Turbo R should do very well but it's even rarer than the IIc Plus.

Edited by JamesD, Sat Dec 1, 2007 7:50 PM.


#29 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,261 posts
  • Location:Australia

Posted Sat Dec 1, 2007 8:26 PM

Anyone got a BBC Model B?

I can remember the BASIC in that machine was pretty quick (2 MHz 6502, and apparently it didn't suffer from any DMA overheads).

Edited by Rybags, Sat Dec 1, 2007 8:26 PM.


#30 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sat Dec 1, 2007 9:22 PM

Anyone got a BBC Model B?

I can remember the BASIC in that machine was pretty quick (2 MHz 6502, and apparently it didn't suffer from any DMA overheads).

The Acorn BASIC was also known to be pretty fast but since it wasn't the fastest machine from the original test we know it took over 1:09.

#31 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sun Dec 2, 2007 1:42 AM

A Speccy emulator came up with the following which should be pretty close:
Time 4:38
Accuracy 0.0006685257
Random 4.322937

#32 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sun Dec 2, 2007 2:07 AM

I just found out there is a BASIC for the IIgs called GSoft BASIC that supports the 65816 and IIgs Toolbox features among other things. I'll see if the demo will run the benchmark.

#33 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,923 posts
  • Location:U.S. -- Midwest

Posted Mon Dec 3, 2007 6:20 AM

I just found out there is a BASIC for the IIgs called GSoft BASIC that supports the 65816 and IIgs Toolbox features among other things. I'll see if the demo will run the benchmark.


Looks like no one is going to come up with a C64 run, so I'll have to get one "out of the closet." ;)

BTW, I remember running another benchmark on an original 4.77 MHz IBM PC against my new Basic XL in FAST mode, and BXL was nearly as fast as the PC. I was impressed by that -- then I ran it on my ST using a third-party Basic and it blew them both away.

-Larry

#34 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,923 posts
  • Location:U.S. -- Midwest

Posted Mon Dec 3, 2007 10:00 AM

Here is an interesting one, I think. I decided to run AHL's Benchmark using Atari Advan Basic.

I was never able to successfully import an Atari Basic SAVEd or LISTed version of the program, even using the "convert" program included. Advan Basic would read the file using LOAD (or LOADS for the LISTED version). After LOADing, it produced no error message, but I could not LIST it to the screen after LOADing, nor would it RUN. (?)

I finally decided it would be quicker to type it in, so I did. After I typed it in, I could LIST it as well as SAVE and then successfully LOAD it from disk and LIST it to the screen. It is possible that my lack of knowledge of Advan Basic caused this, but I did check the manual.

The results:

Time: 456.2 sec
Accuracy .036606
Random 18.92, 11.51

Notes: This was done with regular floating point. The manual states the advantages of using Integers, but Ahl's Benchmark is not really designed for Integers.

-Larry

#35 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Mon Dec 3, 2007 1:32 PM

I have to set up my IIgs and get it running before I can run the test on it.
I also have some other systems I could run the test on.
I have a Panasonic JR-200 which did pretty well in the original test results if I remember right and an NEC Trek.
If I can get the Thomson TO8 I just received up and running I'd like to test it as well.

I would also like to see results for the Amstrad CPC, Enterprise 64/128, Memotech, Ti-99, Lobo Max-80 (fast clock TRS-80 clone) and whatever other 8 bits we can find.

#36 Urchlay OFFLINE  

Urchlay

    Stargunner

  • 1,133 posts

Posted Mon Dec 3, 2007 3:16 PM

...whatever other 8 bits we can find.


Anyone feel like trying to port it to the 2600, with Batari BASIC? The math there is either integer or fixed point... but there's no square root function AFAIK, you'd have to write your own. For A=A^2, you could write A=A*A... though that could be considered cheating (I bet the A8 could run it a lot faster, if you made the same change).

#37 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Mon Dec 3, 2007 4:35 PM

...whatever other 8 bits we can find.


Anyone feel like trying to port it to the 2600, with Batari BASIC? The math there is either integer or fixed point... but there's no square root function AFAIK, you'd have to write your own. For A=A^2, you could write A=A*A... though that could be considered cheating (I bet the A8 could run it a lot faster, if you made the same change).

There were computer expansion modules for some of the video game systems in the original benchmarks.
I know vectrix was one, possibly the Bally arcade...

#38 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Tue Dec 4, 2007 7:28 AM

Dick Smith VZ300

Time 1:55
accuracy .0338745
random 1000 < didn't change rnd(1) to rnd(0).
<edit>
with rnd(0) instead of rnd(1)

Time : 1:48
Accuracy : .0338745
Random : 13.5997

Edited by JamesD, Wed Dec 5, 2007 6:37 AM.


#39 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Tue Dec 4, 2007 7:38 PM

Another interesting link related to benchmarks.
Link

And some benchmarking over on a C128 forum (including some SuperCPU results... yeah it's fast):
Link

Edited by JamesD, Tue Dec 4, 2007 9:15 PM.


#40 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Thu Dec 6, 2007 12:01 AM

* Amstrad CPC 6128 Plus Locomotive BASIC 1.1

Accuracy 3.50475E-05
Random 3.07867885
Time 38.9666667 sec


* benchmarked on WinAPE

Edited by JamesD, Thu Dec 6, 2007 11:57 PM.


#41 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Fri Dec 7, 2007 12:10 AM

I found someone over on a Yahoo TRS-80 group that says they have the actual page with the original benchmarks. He doesn't know where it's at but he promised to keep an eye out for it and to post it when he finds it.

#42 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,923 posts
  • Location:U.S. -- Midwest

Posted Fri Dec 7, 2007 5:39 AM

I found someone over on a Yahoo TRS-80 group that says they have the actual page with the original benchmarks. He doesn't know where it's at but he promised to keep an eye out for it and to post it when he finds it.


That would be great!
-Larry

#43 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sat Dec 8, 2007 7:11 PM

Summary of results so far

* all machines marked with an asterisk were benchmarked on an emulator and results may or may not be accurate


Atari 8 bit----------------------------
Atari Advan Basic 456.2 sec
Accuracy .036606
Random 18.92, 11.51
Atari 8K Basic 405 sec.
Basic XL Time 395.88
Basic XE Time 49.7
Turbo Basic XL 41.6
Turbo Basic Compiler 33.1
Accuracy = .013959
Random = 16.77, 8.80 (actually varies with each run on all versions of BASIC)
Atari Microsoft Basic Time=101.4
Accuracy .150879
Random=2.06506, 2.06506 (5.60 when RANDOMIZEd)


Tandy Color Computer 3 in high speed mode: ---------------------
Time: 71 sec (a=a^2)
44 sec (a=a*a)
With the fast keyboard routine enabled: 69 sec
56 seconds (A=A^2 6309 running in native mode, no 6309 BASIC patches)
34.72 (A=A*A estimate based on 6809 speed increase)
accuracy 5.96284867E-04
random 7.3876276 test1, 24.5601945 test2, 9.40813446 test3
6 seconds (BASIC O9 under OS-9 or Nitros-09)
accuracy 9.45091248E-04


Apple IIc Plus ------------------------
~ 31 seconds (A=A^2 timed by hand)
~18.6 (A=A*A estimated using same speed increase as IIgs)
Apple IIgs 2.8MHz ---------------
Time 42.5 (A=A^2 timed by hand)
25.5 (A=A*A timed by hand)
Accuracy 1.04141235E-03
Random 8.42399192

The Commodore Plus/4 (PAL) ---------------
Time: 109 sec
Accuracy 1.04141235-03
Random 11.1208959


Oric Atmos----------------
Time 144 sec
Accuracy : 1.04141235E-03
Randow : 3.47206211


Dick Smith VZ300 -------------------------------
Time : 108
Accuracy : .0338745
Random : 13.5997


C64 timed by hand -----------------------------------------
A^2 method Time =123 sec.
Accuracy=1.041 e-3
Random 10.09
A*A method Time=76 sec.
Accuracy 3.227 e-4
Random 4.19, 13.83


Sinclair Spectrum* --------------------------------------
Time 290 sec (278 appears to have been a math error)
Time 175 (using "let a=a*a" instead of let a=a^2 and iterative square root instead of SQR) <edit>
Accuracy 0.0006685257
Random 4.322937
ZX Basic TobosFP compiler
a^2 method
time: 12 sec
accuracy: 0.1154175
random:7.373413
a*a method
time: 4.5 sec
accuracy: 0.111023
random: 3.753113 (number is varying)


Amstrad CPC 6128 Plus Locomotive BASIC 1.1 * ---------------------------
Time 38.9666667 sec
Accuracy 3.50475E-05
Random 3.07867885

Edited by JamesD, Mon Dec 10, 2007 11:15 PM.


#44 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,923 posts
  • Location:U.S. -- Midwest

Posted Sun Dec 9, 2007 7:18 AM

For a stock C64 timed by hand:

A*A method
Time=76 sec.
Accuracy 3.227 e-4
Random 4.19, 13.83

A^2 method
Time =123 sec.
Accuracy=1.041 e-3
Random 10.09

There are two sets of times because I originally didn't grasp that the "up arrow" key on the 64 keyboard was the exponent key. So I ran the benchmark using A*A instead of A^2. That produced a much faster time. I'm sure that I did that with one of the Atari Basics, and it produced a slightly slower time.

Accuracy: 1.041 e-3 = .001041
3.227 e-4 = .0003227 Those are correct, aren't they?

-Larry

#45 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,261 posts
  • Location:Australia

Posted Sun Dec 9, 2007 7:29 AM

What's this business with substituting A*A or A^2 for the SQR(A) function ?

SQR is square root - a calculation which takes significantly longer than simply multiplying a number with itself.

#46 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,923 posts
  • Location:U.S. -- Midwest

Posted Sun Dec 9, 2007 7:49 AM

What's this business with substituting A*A or A^2 for the SQR(A) function ?

SQR is square root - a calculation which takes significantly longer than simply multiplying a number with itself.


Hi Rybags-

I don't think that's what I did. Here's the benchmark:

10 ' Ahl's Simple Benchmark
20 FOR N=1 TO 100: A=N
30 FOR I=1 TO 10
40 A=SQR(A): R=R+RND(1)
50 NEXT I
60 FOR I=1 TO 10
70 A=A^2: R=R+RND(1)
80 NEXT I
90 S=S+A: NEXT N
100 PRINT ABS(1010-S/5)
110 PRINT ABS(1000-R)

I did the square root as it said (line 40).

But in line 70, it squares the result which is what I changed from A^2 to A*A.

Am I missing something here? I am presuming that the "up arrow" key on the C64 is the exponent key. Isn't that correct?

-Larry

#47 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,261 posts
  • Location:Australia

Posted Sun Dec 9, 2007 8:19 AM

Soz, my mistake there.

The C-64 has the ^ function - just that it uses the up arrow key (and I can't seem to find just what it's mapped to on the emulator).

From memory, I tried A*A rather than ^2 back in the day on various systems, since it is usually a bit quicker.

Edited by Rybags, Sun Dec 9, 2007 8:20 AM.


#48 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sun Dec 9, 2007 10:56 AM

What's this business with substituting A*A or A^2 for the SQR(A) function ?

SQR is square root - a calculation which takes significantly longer than simply multiplying a number with itself.

What I posted was a typo. Replacing SQR with A*A would not do the same thing.
An iterative replacement for the function that returns the same results would at least be equivalent but much larger but that's how the faster speccy number was achieved.

The numbers for the C64 look a little lower than I remember the C64 having in the original benchmark results.
I keep telling people to run them on a real machine and they keep telling me how perfect the timing on the emulators is.
Funny, I ran some benchmarks on a CoCo 3 emulator and it was actually slower than the real thing.
That's why the asterisk on some results.

Edited by JamesD, Sun Dec 9, 2007 11:01 AM.


#49 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,261 posts
  • Location:Australia

Posted Sun Dec 9, 2007 11:05 AM

Some emulators are fairly rudimentary in their implementation and timing.

But, WinVICE and Atari800Win+ are cycle-exact, and should return results extremely close to the real thing, provided the machine they run on can maintain 100% speed.

#50 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,853 posts
  • Location:Flyover State

Posted Sun Dec 9, 2007 11:07 AM

For a stock C64 timed by hand:

A*A method
Time=76 sec.
Accuracy 3.227 e-4
Random 4.19, 13.83

A^2 method
Time =123 sec.
Accuracy=1.041 e-3
Random 10.09

There are two sets of times because I originally didn't grasp that the "up arrow" key on the 64 keyboard was the exponent key. So I ran the benchmark using A*A instead of A^2. That produced a much faster time. I'm sure that I did that with one of the Atari Basics, and it produced a slightly slower time.

Accuracy: 1.041 e-3 = .001041
3.227 e-4 = .0003227 Those are correct, aren't they?

-Larry

The difference in accuracy makes me nervous. They *should* be the same.




0 user(s) are browsing this forum

0 members, 0 guests, 0 anonymous users