Jump to content

Photo

Ahl's Benchmark?


141 replies to this topic

#1 Larry OFFLINE  

Larry

    River Patroller

  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Thu Nov 22, 2007 9:32 AM

David Ahl, published his benchmark in the November, 1983 issue of his Creative Computing magazine. (The first computer magazine I ever subscribed to.) That text article is posted at the Classic Computer Magazine Archive (November, 1983 issue):
http://www.atarimaga...arison_test.php

But the actual benchmark and the comparison times are not included. The benchmark is posted in several places on the internet, but I can't find the original times. Does anyone have the original article (or link) with the comparison times?

-Larry

Edited by Larry, Wed Nov 28, 2007 6:07 PM.


#2 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Thu Nov 22, 2007 9:48 AM

David Ahl, published his benchmark in the November, 1983 issue of his Creative Computing magazine. (The first computer magazine I ever subscribed to.) That text article article is posted at the Classic Computer Magazine Archive (November, 1983 issue):
http://www.atarimaga...arison_test.php

But the actual benchmark and the comparison times are not included. The benchmark is posted in several places on the internet, but I can't find the original times. Does anyone have the original article (or link) with the comparison times?

-Larry

I've been looking for the code for the benchmark for some time. If you have a link, please post it.
<edit>
Never mind... it didn't turn up on google last time I searched but now it's in several places.

http://www.floodgap....ac/ahl/imp.html
http://setiathome.be.../ahl/basic.html
http://www3.sympatic...tari_basic.html

Edited by JamesD, Thu Nov 22, 2007 9:54 AM.


#3 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Thu Nov 22, 2007 10:03 AM

I've been looking for the code for the benchmark for some time. If you have a link, please post it.


If you Google "Ahl's Benchmark" you'll find several listings. Note it included as a sidebar in the article "Atari Basic -- The Good, the Bad, and the Ugly"

http://www3.sympatic...tari_basic.html

Some listings require very minor changes to run under Atari Basic.

-Larry

#4 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,263 posts
  • Location:Australia

Posted Thu Nov 22, 2007 10:25 AM

Like the article says, Atari BASIC was/is embarassingly slow.

Mainly thanks to half-baked FP ROM, and inefficient line lookup.

In the day, I can remember it usually being between 3rd last to last in the common benchmarks.


Although, if we had something like "MadOnion 2DMark 1985", it would have crushed all comers.

#5 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Thu Nov 22, 2007 10:22 PM

I looked all over the place and couldn't find the benchmark results. I thought it was actually a scan with the article so either the scan was removed from that page or whatever page it was on before is now gone.

#6 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Fri Nov 23, 2007 3:51 AM

Like the article says, Atari BASIC was/is embarassingly slow.

Mainly thanks to half-baked FP ROM, and inefficient line lookup.

In the day, I can remember it usually being between 3rd last to last in the common benchmarks.


Although, if we had something like "MadOnion 2DMark 1985", it would have crushed all comers.


What kind of benchmark (?) was the "MadOnion 2D Mark 1985?"

-Larry

#7 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Fri Nov 23, 2007 3:55 AM

I looked all over the place and couldn't find the benchmark results. I thought it was actually a scan with the article so either the scan was removed from that page or whatever page it was on before is now gone.


I'm speculating, but I'll guess that it was left out through a "zoning" error when the article was OCR'd. I've contacted Kevin to see if by chance he has the original article. IIRC, there were several updates to this article in later issues, but I *do* remember the times being published at some point.
-Larry

#8 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Sat Nov 24, 2007 11:13 AM

We could just run the benchmark on as many machines as we can and post the results here.

If we do that... NO OPTIMIZATIONS!
The loops are spread over several lines intentionally.

#9 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,263 posts
  • Location:Australia

Posted Sat Nov 24, 2007 11:22 AM

What kind of benchmark (?) was the "MadOnion 2D Mark 1985?"

-Larry



It wasn't one.

MadOnion is the old name of Futuremark.

They do PC Benchmarks, the famous ones starting with 3DMark (99, 2000, 01, 03, 05 and 06).

#10 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Tue Nov 27, 2007 7:30 AM

Well, I haven't found anyone with the article yet but I have 1 benchmark that wasn't on the original list.
The Tandy Color Computer 3 in high speed mode:
time 1 min 12 sec
accuracy 5.96284867E-04
random 7.3876276 test1, 24.5601945 test2, 9.40813446 test3

Reruns yielded: 1 min 11 sec.
With the fast keyboard routine enabled: 1 min 9 sec

Random should *not* be evenly distributed (even = too random to really be random) and on TRS-80's you can see the random number generator for better results with:
A=RND(-TIMER)

Not bad results for a Microsoft version of BASIC.
I'm wondering what the faster Atari BASIC replacements managed.

Edited by JamesD, Tue Nov 27, 2007 7:31 AM.


#11 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Wed Nov 28, 2007 6:06 PM

Well, I haven't found anyone with the article yet but I have 1 benchmark that wasn't on the original list.
The Tandy Color Computer 3 in high speed mode:
time 1 min 12 sec
accuracy 5.96284867E-04
random 7.3876276 test1, 24.5601945 test2, 9.40813446 test3

Reruns yielded: 1 min 11 sec.
With the fast keyboard routine enabled: 1 min 9 sec

Random should *not* be evenly distributed (even = too random to really be random) and on TRS-80's you can see the random number generator for better results with:
A=RND(-TIMER)

Not bad results for a Microsoft version of BASIC.
I'm wondering what the faster Atari BASIC replacements managed.


Here are some other results:

Atari 8K Basic Time=405 sec. (!) Accuracy=.013959 Random=16.77, 8.80

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9

Atari Microsoft Basic Time=101.4 Accuracy .150879 Random=2.06506, 2.06506 (5.60 when RANDOMIZEd)

Notes:
The FAST mode of BXL and BXE made essentially no difference in the times, presumably since the line numbers are low and very close together.
Atari Microsoft Basic would not run properly as the benchmark is written. RND(1) gives a random number between 1 and 1. RND(0) appears to work, but gives the same results each time. A RANDOMIZE statement must be added to "seed" the random number generator. The RANDOMIZE value can be taken from POKEY to provide a "random" seed from 0-255 IIRC. Also, the times for AMSB are single precision floating point (not integers).
As usual, TBXL is fastest, with BXE a pretty close second. And what can we say about the speed of Atari Basic...

Can anyone run this on the C64 for comparison?

-Larry

Edited by Larry, Thu Nov 29, 2007 9:33 AM.


#12 Urchlay OFFLINE  

Urchlay

    Stargunner

  • 1,133 posts

Posted Wed Nov 28, 2007 6:15 PM

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9


Wow... I didn't think BXL and BXE were that different, would not have expected the huge speed increase in BXE.

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33


Is this interpreted or compiled Turbo BASIC?

#13 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Wed Nov 28, 2007 6:37 PM

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9


Wow... I didn't think BXL and BXE were that different, would not have expected the huge speed increase in BXE.

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33


Is this interpreted or compiled Turbo BASIC?


Hi Urchlay-

I think this benchmark is all about the math package. BXL uses the regular Atari FP routines. BXE (with the extensions loaded) uses its own "fast math" routines.
I probably ought to run BXE without its extensions to compare that.

This was interpreted TBXL. I probably should run the compiler, also, but I suspect there would not be much difference.

-Larry

#14 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Wed Nov 28, 2007 7:24 PM

The CoCo 3 with a 6309 running in native mode and with no 6309 BASIC patches ran it in 56 seconds. Not bad for an unoptimized Microsloth BASIC. ;)

BTW, the fastest machine that had been benchmarked in the article was the Toshiba T100 with a time of 1:09. The T100 is based on a 4MHz Z80.
Link
<edit>
BTW, a 64180 (Z180) upgrade to the T100 should give about the same # as the 6309.

Edited by JamesD, Wed Nov 28, 2007 7:29 PM.


#15 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Wed Nov 28, 2007 11:09 PM

Here are some other results:

Atari 8K Basic Time=405 sec. (!) Accuracy=.013959 Random=16.77, 8.80

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9

Atari Microsoft Basic Time=101.4 Accuracy .150879 Random=2.06506, 2.06506 (5.60 when RANDOMIZEd)

I just noticed that the Accuracy is lower than other machines.

#16 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Wed Nov 28, 2007 11:10 PM

The Apple IIc Plus completes this in around 31 seconds. I don't have a clock timer on my machine to be more exact.

#17 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Thu Nov 29, 2007 6:59 AM

The Apple IIc Plus completes this in around 31 seconds. I don't have a clock timer on my machine to be more exact.


The IIC Plus was 4 MHz machine, wasn't it? I also wonder what a 65C816 cpu would do with its improved floating point facilities? And likewise a C64 with a SuperCPU?
Or (shudder) a modern PC @3 GHz? It probably would be over before it started... ;)

-Larry

#18 Rybags ONLINE  

Rybags

    Quadrunner

  • 15,263 posts
  • Location:Australia

Posted Thu Nov 29, 2007 7:11 AM

The QBASIC language is on the Win98 install CD.

Pretty sure it works with Windoze XP.

10 rem Ahl's simple benchmark
20 for n = 1 to 100: a = n
30 for i = 1 to 10
40 a = sqr(a): r = r + rnd(1)
50 next i
60 for i = 1 to 10
70 a = a^2: r = r + rnd(1)
80 next i
90 s = s + a: next n
100 print "Accuracy ";abs (1010-s/5)
110 print "Random ";abs (1000-r)

Adapted for QBASIC, with an extra enclosing loop to run the whole thing 100 times (without the printing), does it in about 2 seconds.

Edited by Rybags, Thu Nov 29, 2007 7:18 AM.


#19 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Thu Nov 29, 2007 7:46 AM

The Apple IIc Plus completes this in around 31 seconds. I don't have a clock timer on my machine to be more exact.


The IIC Plus was 4 MHz machine, wasn't it? I also wonder what a 65C816 cpu would do with its improved floating point facilities? And likewise a C64 with a SuperCPU?
Or (shudder) a modern PC @3 GHz? It probably would be over before it started... ;)

-Larry

It had a 4MHz clock but used cache like the Zip Chip if I remember right so it wouldn't be quite as fast as a full 4MHz machine. Still, it's probably the fastest 6502 based system that was made unless a Laser 128EX2 or Franklin can outrun it.

Applesoft Basic was never updated to support the 65816 so a standard speed IIgs will be slower than the IIc Plus. With one of the CPU upgrades it would be faster.
The SuperCPU (?) for the C64 running at 20MHz should be the fastest 65816 unless one of the IIgs upgrades was available at that clock speed.

#20 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Thu Nov 29, 2007 8:35 AM

Well, using BASIC O9 under OS-9 on the CoCo 3 it runs in 6 seconds. :D
But the benchmark was only intended to compare built in BASICs, not compiled BASIC.
Still, it gives you an idea of what difference integers and not having to look up line numbers makes.

#21 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Thu Nov 29, 2007 5:16 PM

The Commodore Plus/4 (PAL?) had these results:

Accuracy 1.04141235-03
Random 11.1208959
Time 1:49

Note: Time was calculated AFTER printing the rest of the results

#22 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Fri Nov 30, 2007 6:39 AM

Well, using BASIC O9 under OS-9 on the CoCo 3 it runs in 6 seconds. :D
But the benchmark was only intended to compare built in BASICs, not compiled BASIC.
Still, it gives you an idea of what difference integers and not having to look up line numbers makes.


Did you revise the benchmark coding to account for integers? I tried Atari Microsoft Basic defining integers as N,I,S. But it had little effect on the times. I presume the accuracy would not be meaningful using integers.

-Larry

#23 Urchlay OFFLINE  

Urchlay

    Stargunner

  • 1,133 posts

Posted Fri Nov 30, 2007 8:12 AM

Did you revise the benchmark coding to account for integers? I tried Atari Microsoft Basic defining integers as N,I,S. But it had little effect on the times. I presume the accuracy would not be meaningful using integers.


I remember reading somewhere that Commodore 64 BASIC's integers are actually slower than its floating point, because the integers have to get converted back to FP any time they're used in calculations. Maybe Atari MS BASIC suffers the same problem?

(Being slower doesn't make them useless: integer variables still use less RAM than floating point variables)

#24 JamesD OFFLINE  

JamesD

    Quadrunner

  • 7,857 posts
  • Location:Flyover State

Posted Fri Nov 30, 2007 8:12 AM

Well, using BASIC O9 under OS-9 on the CoCo 3 it runs in 6 seconds. :D
But the benchmark was only intended to compare built in BASICs, not compiled BASIC.
Still, it gives you an idea of what difference integers and not having to look up line numbers makes.


Did you revise the benchmark coding to account for integers? I tried Atari Microsoft Basic defining integers as N,I,S. But it had little effect on the times. I presume the accuracy would not be meaningful using integers.

-Larry

Someone over on CoCo3.com ran it. Revising the benchmark has good and bad points.
Technically, you aren't doing the same thing. But on the other hand, it shows what you can do in the real world.
<edit> Ahl himself may have modified some versions to use integers. We don't know since he didn't post every version.
Accuracy = 9.45091248E-04
That's less than regular MS BASIC but still better than the Atari.

BTW, the ABS() has limitations that required the use of the if/then/else.
DIM i,n:INTEGER
 s:=0 \ r:=0
 a:=RND(-VAL(RIGHT$(DATE$,2))))
 PRINT DATE$
 FOR n=1 TO 100 \a:=n
   FOR i=1 TO 10
	   a:=SQ(a) \r:=r+RND(0)
   NEXT i
   FOR i=1 TO 10
	  a:=SQ(a) \r:=r+RND(0)
   NEXT i
   s:=s+a
 NEXT n
 PRINT "Accuracy ";
 IF s<5050 THEN
	PRINT 1010-s/5
 ELSE
	PRINT s/5-1010
 ENDIF
 PRINT "Random "; ABS(1000-r)
 PRINT DATE$

Edited by JamesD, Fri Nov 30, 2007 7:06 PM.


#25 Larry OFFLINE  

Larry

    River Patroller

  • Topic Starter
  • 3,931 posts
  • Location:U.S. -- Midwest

Posted Fri Nov 30, 2007 1:37 PM

A small update:

I compiled the benchmark with the Turbo Basic Compiler, and it runs in 33.1 seconds -- nearly 20% faster.

-Larry




0 user(s) are browsing this forum

0 members, 0 guests, 0 anonymous users