From WSMITH@msnet.mathstat.uoguelph.ca  Fri Mar 24 08:23:03 1995
Received: from ccshst06.cs.uoguelph.ca  for WSMITH@msnet.mathstat.uoguelph.ca
	by www.ccl.net (8.6.10/930601.1506) id IAA09310; Fri, 24 Mar 1995 08:19:27 -0500
From: <WSMITH@msnet.mathstat.uoguelph.ca>
Received: from msnet.nw.uoguelph.ca (msnet.mathstat.uoguelph.ca) by ccshst06.cs.uoguelph.ca with SMTP
	(1.37.109.14/16.2) id AA009131245; Fri, 24 Mar 1995 08:20:45 -0500
Received: from MSNET/MAILQ by msnet.nw.uoguelph.ca (Mercury 1.11);
    Fri, 24 Mar 95 8:19:25 GMT-5
Received: from MAILQ by MSNET (Mercury 1.11); Fri, 24 Mar 95 8:19:17 GMT-5
Organization:  Dept. of Math & Stats.
To: chemistry@ccl.net
Date:          Fri, 24 Mar 1995 08:19:11 EDT
Subject:       Re: CCL:Re:  CCL:optimization with constraints
X-Pmrqc:       1
Priority: normal
X-Mailer:     Pegasus Mail v3.1 (R1a)
Message-Id: <18A52BF2A0A@msnet.nw.uoguelph.ca>


Hello,

Eric Martin wrote:
>
>My understanding is that fitting a transformed function might not only
>affect the efficiency, it can also affect the answer.  The least
>squares best fit of the transformed problem will have different
>parameter values (even after the reverse transorm) than the best fit of
>the untransformed (albeit constrained) problem, since the sum of
>squared residuals of a different function is being minimized.  This can
>sometimes be rectified by applying weighted least squares.  See:
>"When, Why and How to Use Weighted Least Squares", Robert de Levie, J.
>Chem. Edu., 63, (1986) p. 10.

    This is a very good point, but it refers to a transformed _model_, 
not to constraints on the model itself (the latter being a purely 
computational issue).

A classic example is

y = A \exp (b*x)  (1)

with additive error.

In the old days (before computers), this would be changed to

\log (y) = A' + bx   (2)
which is now a linear model (in terms of the parameters A and b).

One can now use "good old linear regression" to solve this problem.
However, as Eric correctly points out, this is only (statistically) 
correct if the error is _additive_ in Eqn. (2).  Of course, if the 
error was additive in (1), this surely will not be the case in (2).  
The parameter values resulting from the 2 fits will indeed be 
different.

    One can (partially) compensate for the transformation by 
_weighting_ the residuals in fitting Eqn (2).  In general, the 
weights for a transformation f(y) are given by 

w_i = \frac{1}{f'(y_{i}}^2}

which in the case of (2) becomes 

w_i = \frac{1}{y_{i}^2}

(Of course, the above comments also assume normally distributed 
errors with equal variances, etc.)

    I never cease to be amazed by the fact that regression (and 
parameter estimation) is taught so poorly in universities - 
it's as if computers had never been invented!  There's a _huge_ 
emphasis on linear models and on normally distributed errors, and 
neglect of many important topics (errors in the independent 
variables, nonlinear models, non-normal error distributions, ....  )
Maybe it's related to the fact that beginning stats courses (where 
regression is first taught) are oriented towards the social sciences -
 where there is no such thing as a "fundamental law" and one might as 
well use a linear model.

    Some years ago, I wrote a report discussing the above and other 
problems related to nonlinear parameter estimation.  If anyone would 
like a copy, please send me an E-Mail request.



Best Regards,


W. R. Smith                        Professor
                                   Dept. of Mathematics and Statistics
                                   and School of Engineering 
                                   University of Guelph
FAX: 519-837-0221                  Guelph, Ontario
Tel: 519-824-4120, ext. 3038       CANADA N1G 2W1