From chemistry-request $#at#$ server.ccl.net Wed Apr 21 13:47:43 1999 Received: from www.ccl.net (www.ccl.net [192.148.249.5]) by server.ccl.net (8.8.7/8.8.7) with ESMTP id NAA24217; Wed, 21 Apr 1999 13:47:43 -0400 Received: from ccl.net (atlantis.ccl.net [192.148.249.4]) by www.ccl.net (8.8.3/8.8.6/OSC/CCL 1.0) with ESMTP id NAA28855 Wed, 21 Apr 1999 13:44:20 -0400 (EDT) Received: from bioc1.msi.com (bioc1.msi.com [146.202.0.2]) by ccl.net (8.8.6/8.8.6/OSC 1.1) with ESMTP id NAA08094 for ; Wed, 21 Apr 1999 13:44:17 -0400 (EDT) Received: (from daemon \\at// localhost) by bioc1.msi.com (980427.SGI.8.8.8/970903.SGI.AUTOCF) id KAA00034 for ; Wed, 21 Apr 1999 10:42:45 -0700 (PDT) Received: from dxg-pc.msi.com(146.202.11.19) by bioc1.msi.com via smap (V2.0) id xma000015; Wed, 21 Apr 99 10:42:14 -0700 Message-Id: <3.0.32.19990421104021.00a8f450&$at$&146.202.6.130> X-Sender: dgregory#* at *#146.202.6.130 X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Wed, 21 Apr 1999 10:40:21 -0700 To: chemistry*- at -*ccl.net (Comp. Chem. List) From: Don Gregory Subject: Re: CCL:Benchmarks and software Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Warning: short, perhaps controversial response follows. ; ) >>On Mon, 19 Apr 1999, David Giesen wrote: >> I would like to add a second requirement of all commercial software if I >> may dream a little. > While Peter Shenkin replied ..... >Where there's life, there's hope. :-) A comment on a related statement made earlier, plus a small additional note of my own. It was pointed out that it is *very* difficult to get consensus from any significant number of people, on what 'banchmarks' are acceptable/ useful. This has been my experience, in the extreme. Secondly, people are looking for the commercial companies (note that I am, and have been an employee of MSI for quite some time) to do the benchmarks, and indeed, I can tell you that there has been for many years lively discussion within MSI regarding just that. But the first point keeps rearing its ugly head, and the more we bring in 'consultants' and outside-academics, the more convoluted things get. My additional note is this: there is a 'Consumers Union' and 'Consumers Report' that does analysis and 'benchmarking' for a reason. It is crucial, for them to be useful to the public at large, that such tests be run in an independent, objective, repeatable fashion (the hallmarks of good 'science', eh?) I am *definitely* not suggesting that MSI is shy about having our products tested and compared; we do that evey day as we go into companies world-wide. Of course, evey customer before they buy, wants to know how well you stack up against the competition. I'm not here to soap-box for MSI; I just don't want to give the silly impression that I'm trying to 'hide' where I work. But my main focus is, that given the recent fracas of linus 'benchmarks', one might think that the populace at large might have an interest in *establishing* what good benchmarks are, and how they are run and by what criteria they are judged. I think if that could be done, MSI would be thrilled to compete. But as with any 'reality check' the closer one looks, the more complicated things become. This is not to say that such efforts are not important, but that it isn't as simple as it implies to just say "someone should do it". Don Gregory Dr. Don Gregory (dgregory ^%at%^ msi.com) Asia Pacific Science Liaison Molecular Simulations Inc. 9685 Scranton Rd. San Diego, CA 92121 (619) 799-5331 http://www.msi.com