Ahl's Benchmark?

[Rybags]

C-64s integer variables are actually pretty useless.

 

You only save RAM if they are referenced less than several times.

 

That is because the C-64 doesn't tokenize variable names in the program code - each reference takes a byte per character.

 

So, i costs you 2 bytes minimum each reference, where FP is only one byte.

 

Of course, for arrays the savings will usually outweigh the losses.

[JamesD]

Oric Atmos

 

Time 2:24

Accuracy : 1.04141235E-03

Randow : 3.47206211

 

5 GET A$:SHOOT
10 REM Ahl_s simple benchmark
20 FOR N = 1 TO 100: A = N
30 FOR I = 1 TO 10
40 A = SQR(A): R = R + RND(1)
50 NEXT I
60 FOR I = 1 TO 10
70 A = A^2: R = R + RND(1)
80 NEXT I
90 S = S + A: NEXT N
100 PRINT "Accuracy ";ABS (1010-S/5)
110 PRINT "Random ";ABS (1000-R)
120 SHOOT

 

The SHOOT command was added to make it easier to time.

[JamesD]

Well, as far as 8 bit machines go the current uncompiled best is around 30 seconds for the Apple IIc Plus (4MHz 65C02). The best compiled is 6 seconds for the Tandy CoCo 3 using BASIC09.

 

Any faster from the 6502 will probably require an after market accelerator board. I'm guessing the IIgs would be fastest but the C64 has a 65c816 board with a high clock speed.

 

The MSX Turbo R should do very well but it's even rarer than the IIc Plus.

Edited December 2, 2007 by JamesD

[Rybags]

Anyone got a BBC Model B?

 

I can remember the BASIC in that machine was pretty quick (2 MHz 6502, and apparently it didn't suffer from any DMA overheads).

Edited December 2, 2007 by Rybags

[JamesD]

Anyone got a BBC Model B?

 

I can remember the BASIC in that machine was pretty quick (2 MHz 6502, and apparently it didn't suffer from any DMA overheads).

The Acorn BASIC was also known to be pretty fast but since it wasn't the fastest machine from the original test we know it took over 1:09.

[JamesD]

A Speccy emulator came up with the following which should be pretty close:

Time 4:38

Accuracy 0.0006685257

Random 4.322937

[JamesD]

I just found out there is a BASIC for the IIgs called GSoft BASIC that supports the 65816 and IIgs Toolbox features among other things. I'll see if the demo will run the benchmark.

[+Larry]

I just found out there is a BASIC for the IIgs called GSoft BASIC that supports the 65816 and IIgs Toolbox features among other things. I'll see if the demo will run the benchmark.

 

Looks like no one is going to come up with a C64 run, so I'll have to get one "out of the closet."

 

BTW, I remember running another benchmark on an original 4.77 MHz IBM PC against my new Basic XL in FAST mode, and BXL was nearly as fast as the PC. I was impressed by that -- then I ran it on my ST using a third-party Basic and it blew them both away.

 

-Larry

[+Larry]

Here is an interesting one, I think. I decided to run AHL's Benchmark using Atari Advan Basic.

 

I was never able to successfully import an Atari Basic SAVEd or LISTed version of the program, even using the "convert" program included. Advan Basic would read the file using LOAD (or LOADS for the LISTED version). After LOADing, it produced no error message, but I could not LIST it to the screen after LOADing, nor would it RUN. (?)

 

I finally decided it would be quicker to type it in, so I did. After I typed it in, I could LIST it as well as SAVE and then successfully LOAD it from disk and LIST it to the screen. It is possible that my lack of knowledge of Advan Basic caused this, but I did check the manual.

 

The results:

 

Time: 456.2 sec

Accuracy .036606

Random 18.92, 11.51

 

Notes: This was done with regular floating point. The manual states the advantages of using Integers, but Ahl's Benchmark is not really designed for Integers.

 

-Larry

[JamesD]

I have to set up my IIgs and get it running before I can run the test on it.

I also have some other systems I could run the test on.

I have a Panasonic JR-200 which did pretty well in the original test results if I remember right and an NEC Trek.

If I can get the Thomson TO8 I just received up and running I'd like to test it as well.

 

I would also like to see results for the Amstrad CPC, Enterprise 64/128, Memotech, Ti-99, Lobo Max-80 (fast clock TRS-80 clone) and whatever other 8 bits we can find.

[Urchlay]

...whatever other 8 bits we can find.

 

Anyone feel like trying to port it to the 2600, with Batari BASIC? The math there is either integer or fixed point... but there's no square root function AFAIK, you'd have to write your own. For A=A^2, you could write A=A*A... though that could be considered cheating (I bet the A8 could run it a lot faster, if you made the same change).

[JamesD]

...whatever other 8 bits we can find.

 

Anyone feel like trying to port it to the 2600, with Batari BASIC? The math there is either integer or fixed point... but there's no square root function AFAIK, you'd have to write your own. For A=A^2, you could write A=A*A... though that could be considered cheating (I bet the A8 could run it a lot faster, if you made the same change).

There were computer expansion modules for some of the video game systems in the original benchmarks.

I know vectrix was one, possibly the Bally arcade...

[JamesD]

Dick Smith VZ300

 

Time 1:55

accuracy .0338745

random 1000 < didn't change rnd(1) to rnd(0).

<edit>

with rnd(0) instead of rnd(1)

 

Time : 1:48

Accuracy : .0338745

Random : 13.5997

Edited December 5, 2007 by JamesD

[JamesD]

Another interesting link related to benchmarks.

Link

 

And some benchmarking over on a C128 forum (including some SuperCPU results... yeah it's fast):

Link

Edited December 5, 2007 by JamesD

[JamesD]

* Amstrad CPC 6128 Plus Locomotive BASIC 1.1

 

Accuracy 3.50475E-05

Random 3.07867885

Time 38.9666667 sec

 

 

* benchmarked on WinAPE

Edited December 7, 2007 by JamesD

[JamesD]

I found someone over on a Yahoo TRS-80 group that says they have the actual page with the original benchmarks. He doesn't know where it's at but he promised to keep an eye out for it and to post it when he finds it.

[+Larry]

I found someone over on a Yahoo TRS-80 group that says they have the actual page with the original benchmarks. He doesn't know where it's at but he promised to keep an eye out for it and to post it when he finds it.

 

That would be great!

-Larry

[JamesD]

Summary of results so far

 

* all machines marked with an asterisk were benchmarked on an emulator and results may or may not be accurate

 

 

Atari 8 bit----------------------------

Atari Advan Basic 456.2 sec

Accuracy .036606

Random 18.92, 11.51

Atari 8K Basic 405 sec.

Basic XL Time 395.88

Basic XE Time 49.7

Turbo Basic XL 41.6

Turbo Basic Compiler 33.1

Accuracy = .013959

Random = 16.77, 8.80 (actually varies with each run on all versions of BASIC)

Atari Microsoft Basic Time=101.4

Accuracy .150879

Random=2.06506, 2.06506 (5.60 when RANDOMIZEd)

 

 

Tandy Color Computer 3 in high speed mode: ---------------------

Time: 71 sec (a=a^2)

44 sec (a=a*a)

With the fast keyboard routine enabled: 69 sec

56 seconds (A=A^2 6309 running in native mode, no 6309 BASIC patches)

34.72 (A=A*A estimate based on 6809 speed increase)

accuracy 5.96284867E-04

random 7.3876276 test1, 24.5601945 test2, 9.40813446 test3

6 seconds (BASIC O9 under OS-9 or Nitros-09)

accuracy 9.45091248E-04

 

 

Apple IIc Plus ------------------------

~ 31 seconds (A=A^2 timed by hand)

~18.6 (A=A*A estimated using same speed increase as IIgs)

Apple IIgs 2.8MHz ---------------

Time 42.5 (A=A^2 timed by hand)

25.5 (A=A*A timed by hand)

Accuracy 1.04141235E-03

Random 8.42399192

 

The Commodore Plus/4 (PAL) ---------------

Time: 109 sec

Accuracy 1.04141235-03

Random 11.1208959

 

 

Oric Atmos----------------

Time 144 sec

Accuracy : 1.04141235E-03

Randow : 3.47206211

 

 

Dick Smith VZ300 -------------------------------

Time : 108

Accuracy : .0338745

Random : 13.5997

 

 

C64 timed by hand -----------------------------------------

A^2 method Time =123 sec.

Accuracy=1.041 e-3

Random 10.09

A*A method Time=76 sec.

Accuracy 3.227 e-4

Random 4.19, 13.83

 

 

Sinclair Spectrum* --------------------------------------

Time 290 sec (278 appears to have been a math error)

Time 175 (using "let a=a*a" instead of let a=a^2 and iterative square root instead of SQR) <edit>

Accuracy 0.0006685257

Random 4.322937

ZX Basic TobosFP compiler

a^2 method

time: 12 sec

accuracy: 0.1154175

random:7.373413

a*a method

time: 4.5 sec

accuracy: 0.111023

random: 3.753113 (number is varying)

 

 

Amstrad CPC 6128 Plus Locomotive BASIC 1.1 * ---------------------------

Time 38.9666667 sec

Accuracy 3.50475E-05

Random 3.07867885

Edited December 11, 2007 by JamesD

[+Larry]

For a stock C64 timed by hand:

 

A*A method

Time=76 sec.

Accuracy 3.227 e-4

Random 4.19, 13.83

 

A^2 method

Time =123 sec.

Accuracy=1.041 e-3

Random 10.09

 

There are two sets of times because I originally didn't grasp that the "up arrow" key on the 64 keyboard was the exponent key. So I ran the benchmark using A*A instead of A^2. That produced a much faster time. I'm sure that I did that with one of the Atari Basics, and it produced a slightly slower time.

 

Accuracy: 1.041 e-3 = .001041

3.227 e-4 = .0003227 Those are correct, aren't they?

 

-Larry

[Rybags]

What's this business with substituting A*A or A^2 for the SQR(A) function ?

 

SQR is square root - a calculation which takes significantly longer than simply multiplying a number with itself.

[+Larry]

What's this business with substituting A*A or A^2 for the SQR(A) function ?

 

SQR is square root - a calculation which takes significantly longer than simply multiplying a number with itself.

 

Hi Rybags-

 

I don't think that's what I did. Here's the benchmark:

 

10 ' Ahl's Simple Benchmark

20 FOR N=1 TO 100: A=N

30 FOR I=1 TO 10

40 A=SQR(A): R=R+RND(1)

50 NEXT I

60 FOR I=1 TO 10

70 A=A^2: R=R+RND(1)

80 NEXT I

90 S=S+A: NEXT N

100 PRINT ABS(1010-S/5)

110 PRINT ABS(1000-R)

 

I did the square root as it said (line 40).

 

But in line 70, it squares the result which is what I changed from A^2 to A*A.

 

Am I missing something here? I am presuming that the "up arrow" key on the C64 is the exponent key. Isn't that correct?

 

-Larry

[Rybags]

Soz, my mistake there.

 

The C-64 has the ^ function - just that it uses the up arrow key (and I can't seem to find just what it's mapped to on the emulator).

 

From memory, I tried A*A rather than ^2 back in the day on various systems, since it is usually a bit quicker.

Edited December 9, 2007 by Rybags

[JamesD]

What's this business with substituting A*A or A^2 for the SQR(A) function ?

 

SQR is square root - a calculation which takes significantly longer than simply multiplying a number with itself.

What I posted was a typo. Replacing SQR with A*A would not do the same thing.

An iterative replacement for the function that returns the same results would at least be equivalent but much larger but that's how the faster speccy number was achieved.

 

The numbers for the C64 look a little lower than I remember the C64 having in the original benchmark results.

I keep telling people to run them on a real machine and they keep telling me how perfect the timing on the emulators is.

Funny, I ran some benchmarks on a CoCo 3 emulator and it was actually slower than the real thing.

That's why the asterisk on some results.

Edited December 9, 2007 by JamesD

[Rybags]

Some emulators are fairly rudimentary in their implementation and timing.

 

But, WinVICE and Atari800Win+ are cycle-exact, and should return results extremely close to the real thing, provided the machine they run on can maintain 100% speed.

[JamesD]

For a stock C64 timed by hand:

 

A*A method

Time=76 sec.

Accuracy 3.227 e-4

Random 4.19, 13.83

 

A^2 method

Time =123 sec.

Accuracy=1.041 e-3

Random 10.09

 

There are two sets of times because I originally didn't grasp that the "up arrow" key on the 64 keyboard was the exponent key. So I ran the benchmark using A*A instead of A^2. That produced a much faster time. I'm sure that I did that with one of the Atari Basics, and it produced a slightly slower time.

 

Accuracy: 1.041 e-3 = .001041

3.227 e-4 = .0003227 Those are correct, aren't they?

 

-Larry

The difference in accuracy makes me nervous. They *should* be the same.