Article:		Developer Notes from WWDC 2003
Subject:		Performance?
Date:		2003-06-24 11:14:48
From:		anonymous2
OK. We all know that benchmarking is a black art, YMMV, etc. However, I have to ask how seriously to take the critique of Apple's performance claims one can read at http://www.haxial.com/spls-soapbox/apple-powermac-G5/ I'm not interested in bashing (WIntel nor Mac), but in whether the apparently substantive issues raised by the above article are valid. E.g. whether the use of a tweaked malloc, non-use of hyperthreading, etc. could be legitimately called benchmark biasing.

Showing messages 1 through 10 of 10.

• Performance?
  2003-06-25 08:29:38  anonymous2 [View]
  
  
  Does anybody understand why the new G5 is only a few percent faster than the 32 bit Dell system?
  The Mac is a 64 bit machine, with a really fast bus and ran applications that were optimized and compiled for the 64 bit processor on a 64 bit operating system.
  Why the hell did it not completely knock down the Dell?
  What happens with the G5 if the first 64 bit Xeon appears? Looks like it will fall behind the intel processors again soon, doesn't it?
• Performance?
  2003-07-02 16:04:11  anonymous2 [View]
  
  
  64 bits is more about address space than speed. It lets you use more RAM; it doesn't necessarily make processing any faster, and might even make it slower in some circumstances. There are a number of articles on Ars Technica (http://www.arstechnica.com/) that discuss this issue in more detail, if you're interested.
  Also, remember the Dell had processors with a clock speed 50% higher than the G5.
  The most important point about the benchmarks, whether they're completly accurate or not, is that they show this new generation of Macs has an architecture and processor that can hold its own with the best the Wintel world can put up. That's huge. All that, and they don't cost an arm and a leg, either.
• Performance?
  2003-07-07 06:28:08  anonymous2 [View]
  
  
  OK, I understand that.
  
  But 64 bit means also twice the data path and thus the possibility to process twice the data in parallel, doesn't it?
  So I would expect roughly twice the performance of a 32 bit system and not only a few percent more speed.
  
  But we also learned in the meantime that the G5 ran on OS 10.2.7. So there was no 64 bit OS running.
  There is also the bottleneck of slow memory that could not catch up with the speed of the bus.
  Michael

• Performance?
  2003-06-25 07:02:54  anonymous2 [View]
  
  
  The guy has an axe to grind. He's ported his apps to OSX but got irritated by "unkind" reviews of his apps by "Mac zealots". Not liking zealotry of any kind I felt some sympathy for him. That is until I tried to use Haxial Text Edit. Now I understand the complaints. An App ported to the platform with the minimum of effort, buggy and no attempt at modification to make it behave like other OSX applications. His attempt to portray himself as a "Mac User" is disingenuous. I've no doubt he does test his apps on OSX, but my impression is of someone who thought they could make a quick buck from Mac users rather than somone who is enthusiastic about or committed to the platform.
  
  He makes some valid criticisms of the way Apple has used SPEC, but is extremely one sided. For example, if SPEC gave any advantage to AltiVec performance I could understand his points on SSE2. It doesn't, and he has since admitted he was wrong on Apple disabling SSE2 anyway. Don't take this seriously. Wait until the G5 comes out and look for serious less partisan reviews of performance on applications that you want to use.
• Performance?
  2003-06-25 10:24:59  anonymous2 [View]
  
  
  I totally agree. While I think the benchmarking was a marketing fiasco on Apple's part. Jobs skewing the numbers again. (when will he realize people will ALWAYS find the truth faster than he can cover up his tricks?)
  
  I ALSO think that those benchmarks, regardless of how skewed they are, are pretty close to useless. The real test is "how fast can you perform the necessary tasks to get your work done".
  
  Wait for some 3rd party benchmarks on specific applications or... just go with what you know. :x
  
  I, for one, think it's funny that anybody would care about the difference, like they were going to switch platforms to get their work done. Heh. If you're experienced with OS9/X or Windows, what are the chances you're gonna switch platforms for a little more speed? Your job will inevitably take longer til you become as comfortable with the new platform as you were with the old.
  
  The only way that's gonna happen is with a personal preference change really. Where you just give up one OS for the other.
  
  Neura

• Performance?
  2003-06-24 22:56:10  anonymous2 [View]
  
  
  http://www.dell.com/us/en/esg/topics/power_ps3q02-khalid.htm
  
  Hint: read the section titled "Testing performance with SPEC CPU2000", it's very very enlightening.

• Performance?
  2003-06-24 16:15:56  anonymous2 [View]
  
  
  i'm not a programmer of developer and don't know much about SPEC and the difference b/n compilers. but, i've looked through the veritech pdf of Apple's tests and apple wins some tests and loses others. the fact that apple published its wins is not offensive at all. that's called marketing...so in a sense, their claim to be the fastest is partially right. informed people can make their own decisions...

• Performance?
  2003-06-24 12:52:42  anonymous2 [View]
  
  
  As far as I know, official SPEC tests have some extensive rules to avoid benchmark cheating. So I guess, that as long as SPEC.org doesn't sue Apple, you could safely say that the benchmarks were done correctly.

• Performance?
  2003-06-24 12:32:19  anonymous2 [View]
  
  
  Let me ask you this: Should we take the Dell benchmarks any more seriously (ie: the ones not measured by Apple/Veritest?) Why isn't the author of that article questioning the Dell benchmarks?
  
  The man telleth a one sided story, methinks...
• RE: Performance?
  2003-06-24 12:45:05  Derrick Story | [View]
  
  
  Actually, a bunch of us spent quite a bit of time talking about the benchmarks last night. We all know they're a "grain of salt" kind of thing. But in the end, we were pretty satisfied with the presentation.
  
  It was definitely much better than in the past when all we saw was a Photoshop comparison.