From chemistry-request@www.ccl.net  Thu Feb 25 09:53:37 1999
Received: from foreman.ac.rwth-aachen.de (foreman.ac.RWTH-Aachen.DE [134.130.100.31])
        by www.ccl.net (8.8.3/8.8.6/OSC/CCL 1.0) with ESMTP id JAA03207
        Thu, 25 Feb 1999 09:53:26 -0500 (EST)
Received: (from landrum@localhost) by foreman.ac.rwth-aachen.de (AIX4.2/UCB 8.7/8.7) id PAA27910; Thu, 25 Feb 1999 15:53:03 +0100 (NFT)
Date: Thu, 25 Feb 1999 15:53:03 +0100 (NFT)
From: "Gregory A. Landrum" <landrum@foreman.ac.rwth-aachen.de>
To: chemistry@www.ccl.net
Subject: Re: CCL:Summary: AMD-K6/linux for G98
In-Reply-To: <Pine.A41.4.05.9902240905370.73636-100000@asterix.uni-muenster
 .de>
Message-ID: <Pine.A41.4.00.9902251543330.24540-100000@foreman.ac.rwth-aachen.de>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII





Normally I stay silent on these things, but this comment was just a bit
too indicative of general "benchmark abuse".

> For pinene using DFT/B3LYP with large basis sets, for
>example, the machine i bought for $11,000 was faster than a Cray T3E-600
>with 4 processors and only 15% slower than a Cray T3E-1200 with 4
>processors, both running parallel Gaussian 94. (The G94 calculations were
>done by Sosa and Frisch, so were probably pretty well optimized for the
>machine, see last July 15 number of J. Comput. Chem. Sosa et al. Vol 19,
>p. 1053). I haven't priced those machines but they must be >$100k. If you
>want to benchmark against one of your own machines, Pulay would be happy
>to do some calculations for you.

The comparison here is *completely* invalid.  If you want to compare
the speed of machines, you must be running the same program.  In this
case, a highly-optimized DFT implementation (the one from Pulay) is being
compared to an implementation which can charitably be called "not so
great", this does not say a single thing about the relative speeds of the
computers running those programs.  

Having Pulay do a benchmark for you would be altering two variables at the
same time:
 (a) the hardware
 (b) the software
it is impossible to decouple the effects of the two changes.
One more time, for good measure:
If you want reliable benchmarks of *hardware* performance, you must
compare the same programs on the different hardware.

-greg

-- 
---------------------
Dr. Greg Landrum  (landrum@foreman.ac.rwth-aachen.de)
Institute of Inorganic Chemistry
Aachen University of Technology
Prof.-Pirlet-Str. 1, D-52074 Aachen, Germany
Phone: 049-241-80-7004
Fax: 049-241-8888-288



