Benchmarking...

[+DarkLord]

As has been shown over and over, ALT (TT, FAST) RAM does

make a significant difference.

 

Dang well gonna have to get some for my Pak 68/3 equipped STacy.

 

She deserves it. 

 

[vol]

18 hours ago, MasterMotorola said:

The CaTTamaran was an accelerator made here in Canada back in 1994 by Cybercube Research Limited (sometimes referred to as CyRel).  It is a small board that is fitted to the ST-RAM card and connects to various pins on the mainboard via wires.  It does not have its own RAM, it's only function is to increase the clock cycles of the 68030 processor and FPU by 150% (48MHz).  The speeds that mdivancic reported at 32Mhz and 48Mhz confirm that my CaTTamaran is either not functioning or hooked up properly.  It was pre-installed in the machine when I bought it 20 years ago and it's possible it has never been working for me.

Thank you for this information.  It seems that mdivancic's CaTTamaran shows rather odd results.  I suspect that the CaTTamaran somehow affects normal timings.  So results from your hardware would be very interesting.

 

12 hours ago, ParanoidLittleMan said:

And still does not seem that understanding influence of cache, it's size, size of RAM used by running SW ...

I recommend to do tests on machines with 68020/30 with cache off and cache on. That's really easy to solve.

No, please don't disable cache!  My pi-spigot implementation is small enough to fit the internal 68020/30 cache.

6 hours ago, mdivancic said:

The CaTTamaran is working fine. I get the expected results when I run GemBench or other programs. I have 16mb of TT ram and 4 mb of ST ram. Plus I can switch between 32/48 without a problem. In fact you can see a speed increase in the results above. Also the CaTTamaran doesn’t boast the TT’s video speed. I do need to pull my TT apart and do some work. I need a new RTC battery and I want to print a new hard drive bay cover so I can access the SD card in my SCSI2SD. I can double check the install at this time. 

 

I ran the test again with the program set to run in TT Ram.

TT@48mhz w/clean boot. PI-ST30.TOS:

  100 - 0.05

  1000 - 1.45

  3000 - 10.99

Thank you. However your results are very strange.  Your previous results for the TT CaTTamaran@48Mhz for PI-ST30 was 11.67 (NVDI) or 12.54s.  How does it become 10.99s now?!  Or you disabled TT RAM the previous time, didn't you?  I also can't understand why your results from the plain TT@32MHz is slower than old moulinaie's results?  So if it is possible please run PI-ST and PI-ST30 on the same machine configuration.  I can suggest the next list of configurations:

1) TT@32MHz;

2) TT@32MHz TT RAM;

3) TT@48MHz;

4) TT@48MHz TT RAM.

 I have also attached the newest binaries to this post but the speed must be the same as for v6.

pi-st-8-beta.zip

[ParanoidLittleMan]

This attitude helps not. "No, please don't disable cache!  My pi-spigot implementation is small enough to fit the internal 68020/30 cache. "

Then, " I also can't understand why your results from the plain TT@32MHz is slower than old moulinaie's results? "

Of course you don't understand.

I wrote in beginning that do not use screen operations during test. And we see very well their influence in latest test s- when it is faster with NVDI for some 8 %. 

If aim is to target CPU's and computers math speed then should perform only math operations, not printing thousands of numbers on screen .

Then, even if code is short, and fits in small cache of 68030 it does not mean that there is no RAM access to larger RAM area - data, stack .

To have some clue of some systems speed we need to know:

CPU clock. RAM speed - and it can be 2 RAMs - so called ST RAM (that's actually shared RAM with video) and Fast RAM. Cache size.

And funny thing is that even if no screen output, running in ST RAM will cause slower work - depending from used screen res.

Anyone posted in what res ran test on TT ?

I proposed to run tests with cache off just to see difference - and in some cases it can be not so big, and I talked about it already.

All in all, I'm done with this thread. Some people prefer ego instead learning.

 

[vol]

2 hours ago, ParanoidLittleMan said:

This attitude helps not. "No, please don't disable cache!  My pi-spigot implementation is small enough to fit the internal 68020/30 cache. "

Then, " I also can't understand why your results from the plain TT@32MHz is slower than old moulinaie's results? "

Of course you don't understand.

I wrote in beginning that do not use screen operations during test. And we see very well their influence in latest test s- when it is faster with NVDI for some 8 %. 

If aim is to target CPU's and computers math speed then should perform only math operations, not printing thousands of numbers on screen .

Then, even if code is short, and fits in small cache of 68030 it does not mean that there is no RAM access to larger RAM area - data, stack .

To have some clue of some systems speed we need to know:

CPU clock. RAM speed - and it can be 2 RAMs - so called ST RAM (that's actually shared RAM with video) and Fast RAM. Cache size.

And funny thing is that even if no screen output, running in ST RAM will cause slower work - depending from used screen res.

Anyone posted in what res ran test on TT ?

I proposed to run tests with cache off just to see difference - and in some cases it can be not so big, and I talked about it already.

All in all, I'm done with this thread. Some people prefer ego instead learning.

Thank you very much.  You message has clarified the situation.  I felt that something changed but I didn't know what is this about.  Instead of a friendly discussion I somehow provoked a rather strange attitude. It is odd.  Sorry I haven't still known what was done wrong by me.
You wrote some criticism about my program and even something personal.  But your points are not true.  Let me explain why.
1) My program is a multi-platform optimized calculator of the number pi.  So it can't help but print numbers.  This is its main purpose.  The benchmarking is only a supplementary purpose.
2) Time of printing is a small fraction of the whole time of the program execution when we calculate 3000 digits.  It suggests that we don't slow down screen output intentionally.  Actually it is not affect results much as someone can things but makes them not typical and provocative.  This is because of times of pure math and screen printing may be shown separately.  So everyone can get an impression of two system characteristics: the CPU speed and speed of character printing.  Of course, screen printing time is excluded from ER value calculation, only the pure math timings are use there.
3) the last results are strange.  So I asked for more details.  However I got your strange post instead of them.  So it means that something is wrong and it is not just about hardware.
4) The influence of cache can be interesting for some research but it is not my case.  Indeed it would be interesting to get these results too but it is a pure curiosity for me.  I only gather data from a typical working mode.
5) You are right that TT RAM makes things faster even without the screen printing.  So I ask people to run tests for both cases to get exact values about the advantage of the presence of Fast RAM.
6) Indeed different screen modes affect timings.  But I assume that people would send the best results using most fast screen modes.  Anyway this doesn't affect pure math timings.  However it is better to get data from the fastest modes because this can provide several more percent of accuracy.
7) You wrote something about an ego.  Sorry I missed you there.  What did you want to say?
It seems that you have shown an unfriendly attitude.  I hope to be wrong.  I haven't deserve this.  I also hope my arguments deserves at least a detailed reply. The PDP-11 people once were attacking my project, they wanted better results for their platform but the Atari ST series shows quite decent results.  So I am really baffled by your position.  

Edited May 27, 2021 by vol

[ParanoidLittleMan]

" Of course, screen printing time is excluded from ER value calculation, only the pure math timings are use there."

Then how on Earth results with NVDI are better for some 8 % than with TOS screen draw ?

[mdivancic]

2 hours ago, vol said:

Thank you very much.  You message has clarified the situation.  I felt that something changed but I didn't know what is this about.  Instead of a friendly discussion I somehow provoked a rather strange attitude. It is odd.  Sorry I haven't still known what was done wrong by me.

We all speak different languages here, but communication is through English. Sometimes we can be unclear.  I don’t get up tight and took no offense by what you posted.

 

First, why would you assume that a user would run at the screen resolution and memory configuration that would give the best results? I ran all my test at TT Medium resolution (and will continue to do so) because that’s what I use for my every day setup. There are six resolutions that you can run on the TT, I do not have the inclination or desire to run several test in each resolution. 
 

On the TT there are three flags you can set for every compatible program, Fastload, TT-Ram (Mem) and TT-Ram (Prg). Test should be run with various combinations of these flags set just for a reference. They are a pain in the but to set and most of the time I forget to do it.

 

Screen Resolutions are definitely impacting results with V6. I did a quick test this morning and I found I get different results with each screen resolution. Because the TT video is not accelerated by the CaTTamaran it has a bigger impact on the results.

 

When I get time I’ll run some more test. I will use the current version. 

[calimero]

I would like to see results from Amiga 500 in 6 bitplane video mode...  

[vol]

On 5/27/2021 at 1:13 PM, ParanoidLittleMan said:

" Of course, screen printing time is excluded from ER value calculation, only the pure math timings are use there."

Then how on Earth results with NVDI are better for some 8 % than with TOS screen draw ?

I wrote afore that those results are very unusual.  Results from the Falcon show only 1% difference.  Anyway I am still baffled by your position.  I am just gathering data.

On 5/27/2021 at 3:45 PM, mdivancic said:

We all speak different languages here, but communication is through English. Sometimes we can be unclear.  I don’t get up tight and took no offense by what you posted.

 

First, why would you assume that a user would run at the screen resolution and memory configuration that would give the best results? I ran all my test at TT Medium resolution (and will continue to do so) because that’s what I use for my every day setup. There are six resolutions that you can run on the TT, I do not have the inclination or desire to run several test in each resolution. 
 

On the TT there are three flags you can set for every compatible program, Fastload, TT-Ram (Mem) and TT-Ram (Prg). Test should be run with various combinations of these flags set just for a reference. They are a pain in the but to set and most of the time I forget to do it.

 

Screen Resolutions are definitely impacting results with V6. I did a quick test this morning and I found I get different results with each screen resolution. Because the TT video is not accelerated by the CaTTamaran it has a bigger impact on the results.

 

When I get time I’ll run some more test. I will use the current version. 

What do you mean in your phrase "I don’t get up tight and took no offense by what you posted"?  It sounds weird for me.  I am just thankful to you because of your help.

I assume that a user picks up the best performance screen mode because otherwise we get provocative results.  I publish them and people will blame me that I show the Atari ST series in the wrong color.   And we have a benchmark, a test for the performance - it is quite natural to get its maximum.

I just asked for help.  If you are not interested you just ignore this request.

Your CaTTamaran results are very interesting.  I am going to start a new thread about them.  You know the Amiga results show that the PI-ST30 main loop is 4 cycles faster than the PI-ST main loop.  But your results show no difference.  I can't understand how this is possible.

On 5/29/2021 at 7:41 AM, calimero said:

I would like to see results from Amiga 500 in 6 bitplane video mode...  

Some people called me an Atari troll.  You know that the Amiga character output routine is slower that this routine for the Atari ST series.  The Amiga WB doesn't have two-color mode, so all tests for this computer were done under the standard 4 colors.

[calimero]

^
hm... Vol, do you know that Amiga CPU is stalled in 6 bit plane mode?

 

I did not refer to slower Amiga “character output”.

 

And it wont make a difference if Amiga would allow 2bit color mode - speed of Pi would be same as 4bit plane mode. 

Edited June 4, 2021 by calimero

[ParanoidLittleMan]

You came here with very low knowledge about Atari ST computer family (what includes up to Falcon). "I'm just gathering data" - no, you claim incorrect things. And even if diff is only 1 % it means that your approach is wrong.

Again, for last time - you should not do printing on screen during math test.

Saying that printing time is excluded from test time - no way sir, pardon monsieur . I know that you use Timer C for measuring test time, and it is 200 Hz. While 1 character print can be in much less time - sorry, but I'm programming expert in this things. And I'm sick of this attitude ... Adieu ...

[vol]

1 hour ago, ParanoidLittleMan said:

You came here with very low knowledge about Atari ST computer family (what includes up to Falcon). "I'm just gathering data" - no, you claim incorrect things. And even if diff is only 1 % it means that your approach is wrong.

Again, for last time - you should not do printing on screen during math test.

Saying that printing time is excluded from test time - no way sir, pardon monsieur . I know that you use Timer C for measuring test time, and it is 200 Hz. While 1 character print can be in much less time - sorry, but I'm programming expert in this things. And I'm sick of this attitude ... Adieu ...

What am I claiming?  What is my approach?  Would you like to clarify your point?  I have never said that I know the Atari ST family very well.  So I missed your point about my very low knowledge.   It seems like the use of a label.  

Why should the results have deviations less than 1%?  It is impossible for some architectures.  Sorry but I have to repeat this program is not just a math test.  It is an optimized multiplatform calculator.  So it MUST print because it is a calculator.

Why do you insists that "Saying that printing time is excluded from test time - no way sir".  It is quite easy to prove that your point is not correct.  Just set IO=0 in the sources, compile and run.  This gives you almost the same results, the deviation is less than 1% for the Atari ST.  Please provide exact numbers if you want to continue your criticism.

And I don't still understand why do you use rather unfriendly phrases like "I'm sick of this attitude" - I am a professional programmer, just let us discuss things.  What can change the fact that "1 character print can be in much less time" for the results?  Would you like to be less cryptic?

It seems that you don't want to have a real discussion.  It is your right but you attacked my project and I had to reply.

BTW I must say thank you to people who helped me: Darklord, moulinaie, MasterMotorola, mdivancic.  Sorry I don't understand what did cause the change in attitude.  

[ParanoidLittleMan]

Typical - ignoring what is written, even asked for concrete answer - I asked to explain diff. of 8 % in result when screen accelerator was used - and did you answer ? No. It is easier to play being offended. Then using offense is best defense attitude - 'attacked project' - LOL . I pointed on some obvious mistakes, and no, 'professional programmer' will never say ' I was wrong' . That's your main problem. Or rather, refusing to learn would be it.  And don't bother to reply. I don't care anymore. And  have better things to do for retro community. Plenty of it.

Btw. influence of printing on screen depends a lot of how much is set.  Then, if it must to print on screen every calculation (result), that can be done after speed test, from RAM, where can store it 'little' faster than printing on screen via not so fast OS calls.

 

[vol]

6 minutes ago, ParanoidLittleMan said:

Typical - ignoring what is written, even asked for concrete answer - I asked to explain diff. of 8 % in result when screen accelerator was used - and did you answer ? No. It is easier to play being offended. Then using offense is best defense attitude - 'attacked project' - LOL . I pointed on some obvious mistakes, and no, 'professional programmer' will never say ' I was wrong' . That's your main problem. Or rather, refusing to learn would be it.  And don't bother to reply. I don't care anymore. And  have better things to do for retro community. Plenty of it.

Btw. influence of printing on screen depends a lot of how much is set.  Then, if it must to print on screen every calculation (result), that can be done after speed test, from RAM, where can store it 'little' faster than printing on screen via not so fast OS calls.

 

 You just tell me how to write my program! It is funny. You can also say that a fish must look like a mouse. Better try to make code for pi-spigot faster at first.  Sorry but I have also to stop replying to your strange claims because you don't read my responses to them.  I repeated my points several times...  But you ignored most of them.   

[MasterMotorola]

My CaTTamaran is working again (it seems a connector came loose after the 6000km trip from the east coast last year).  Running the program in TT-RAM made no difference on the numbers and this was just TOS with nothing else running.  Here are the stats for the latest version on an Atari TT030 @ 48Mhz...

 

Pi-TOS30v8: 0.04, 1.63, 12.57 (100, 1000, 3000 places)

[mdivancic]

7 hours ago, MasterMotorola said:

My CaTTamaran is working again (it seems a connector came loose after the 6000km trip from the east coast last year).  Running the program in TT-RAM made no difference on the numbers and this was just TOS with nothing else running.  Here are the stats for the latest version on an Atari TT030 @ 48Mhz...

 

Pi-TOS30v8: 0.04, 1.63, 12.57 (100, 1000, 3000 places)

Very similar to my runs…

[calimero]

Maybe you can add Inmos Transputer score in your table

 

https://gtello.pagesperso-orange.fr/transputer.htm (almost on the bottom: PI calculation program)